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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

CORRECTNESS  MODULE 


INTRODUCTION  TO  CORRECTNESS 

The  Quality  Factor  Correctness  represents  a  measure  of 
the  extent  to  which  a  program  satisfies  its  specifications  and 
fulfills  the  user’s  mission  objectives.  FIGURES  Co  -  1 
through  Co  -  4  represent  the  hierarchy  of  Metric,  Criteria 
and  Quality  Factor  worksheets  associated  with  Correctness  in 
Requirements  Analysis,  Preliminary  Design,  Detailed  Design 
and  Implementation  phases  of  the  Software  Development  Life 
Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets,  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Correctness  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 
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STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 


Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block -Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(S.l)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


(5.2) 

(5.3) 

(5.4) 

(5.5) 

(5.6) 

Step  6  - 
(6.1) 

(6.2) 

(6.3) 

(6.4) 

(6.5) 


Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet . 

Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

FACTOR  WORKSHEETS 

Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 


(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required^ 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


Co-8 


HOW  TO  USE  SCORE  CHART  SET^ 


When  you  apply  worksheets  to  system/subsystem/module, 
you  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  .the  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  -  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 
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Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


i 

I 
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I 
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Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set’s 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 
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RBQUIRBCNTS 

ANALYSIS 


ESD  PRODUCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  I  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPC I 

e  Product  B.l  -  Possible  higher-level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor- supplied 
information  should  be  searched. 
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SET 
TO  BE 
APPLIED 


ESP  PRODUCTS  FOIl  DESKfl  PHASE 

Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 

e  Product  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

e  Product  B.  :  Preliminary  Product  Specifications, 

including  the  above 

e  Product  D.  CDR  minutes  4  action  item  responses 


Product  D. 


Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note, 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  information 

requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 


CODE 


ALTERED  PRODUCT 


ESP  PRODUCTS  FOR  POPING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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INTEGRATION  PHASE 


Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPGI  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system- level 
Test  Procedures 

B-D 

e  Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  Product  2:  Computer  Progrpm  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changes. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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□□ 


COMPLETENESS  CHECKLIST 


Form  Code:  CoRAM.l 


LIFE  CYCLE  PHASE: 

REQUIREMENTS  ANALYSIS 


NA»C: _ 


DATA  COLLECTION  WORXSECTION: 


sc me 


1.0  Ikiambi 


References  (2) 


1.1  Are  requirements  itemized  so  that  the 
various  functions  to  be  performed, 
their  inputs  and  outputs,  are  clearly 
delineated? 

Yes  -  1,  No  •  0 

2.0  External  Data  Reference  (3) 

2.1  Nuaber  of  data  references  which  are 
defined. 

2.2  Number  of  major  data  references. 


Score  »•  12.11  «►  1 2.2 1 
3.0  Major  Functions  Used  (4) 

3.1  Number  of  defined  functions 
used. 

3.2  limber  of  functions  identified. 
Score  ■  l3.l|  t  1 3I2] 

4.0  Major  Functions  Defined  (5) 

4.1  Number  of  identified  functions 
*•  defined. 


4.2  Nuaber  of  functl 


Scote 


■  of  functions  it 

•  HU  E 


identified. 


PREPARED  BY: 


APPROVED  BY: 


COMPLETENESS  QffiOg.TfiT 


Fora  Code:  CoRAM.  1 


CYCLE  PHASE: 

REQUIREMENT  ANALYSIS 


ISOURCE(S) : 


DATA  COLLECTION  WORKSECTION  (continued) : 


5.0  Decision  Points  Defined  (6) 

5.1  Is  the  flow  of  processing  and  all 
decision  poises  in  that  flow  defined? 

Yes  -  1,  No  -  0 


6.1  taaber  of  defined  and  referenced 
calling  sequence  parameters  that 
agree  between  fractions. 

6.2  Ifcrter  of  calling  sequence 


7.0  Preble 


Score  ■  16.11  ♦  16,21 
am  Reports  Resolved  (8) 


7.1  Naber  of  those  problem  reports  that 
have  bean  dosed  (resolved). 

7.2  Nunfcer  of  problem  reports  related 
to  the  requirements  that  have  been 
reported. 


Score  ■  EH  *  EH 


METRIC  WORKSECTION: 


Metric  Value 


Sum  of  Above  Scores 
hio.  of  Bata  Elementi 


III.  EVALUATION  WGRKSECTICN:  What  is  your  evaluation  of  the 
reviswed  products  based  on  the  data  elements  above? 
(1-10) _ (8  If  you  are  unable  to  evaluate) 


□□ 


CRITERIA  SUMMARY  WCRKSECITON 


1.  Conpleteness 


II.  FACTOR  RORKSECITCN: 


III.  EVALUATION 


Factor  Value  • 


Sun  of  Above  Scores 


ON:  feat  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (3  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: _ 


^  APPROVED  BY: 

■e  m 

m  PA3E: 

Co-19 


□□I 


COMPLETENESS  OffiCKLIST 


Form  Code:  Co  PPM.  1 


|UEE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


[J  SY5TB4 
SUBSYSTEM 
MODULE 


NAME: 


SOURCE (S) : 


I. 


DATA  COLLECTION  WCRKSECTION: 

1.0  lhaehiguous  References  (2) 

1.1  Are  requirements  itemized  so  that  the 
various  functions  to  be  performed, 
their  inputs  and  outputs,  are  clearly 
delineated? 

Yes  •  1,  No  ■  0 

2.0  External  Data  Reference  (3) 

2.1  Nunfcer  of  data  references  which  are- 
defined. 

2.2  Nmber  of  major  data  references. 

1.2.2 1 


Score  - 


2.1 


3.0  Major  Functions  Used  (4) 

3.1  Number  of  defined  functions 
used. 

3.2  Nuber  of  functions  identified. 
Scote  • 


3.1 


3.2 


4.0  Major  Functions  Defined  (5} 

4.1  Niaber  of  identified  functions 
defined. 

4.2  Nueber  of  functions  identified. 

*  EH 


Score  ■ 


4.1 


PREPARED  BY:. 
HATE: _ 


APPROVED  BY: _ 

DATE: _ 


YN 


SCORE 


□ 


□ 


□ 


□ 


Co -20 


COMPLETENESS  CHECKLIST 


Form  Code:_ 

CoPEM.l 

SCUO(S): 

■■S 

0  SYSTEM 

NM£: 

□  SUBSYSTEM 

□  MODULE 

5.0  Decision  Points  Defined  (6) 

5.1 

Is  the  floe  of  processing  sad  all 

PM 

decision  points  in  that  floe  defined? 

Q 

m 

— i 

Yes  -  1,  No  •  0  • 

i _ i 

6.0  Agrei 

■MHt  of  CelUne  Sm 

m nr*  Panvens  (7) 

6.1 

calling  sequence  parameters  that 

— 1 

agree  between  fractions. 

_ 1 

6.2 

Msfcer  of  celling  sequence 

psmvtors* 

□ 

Score  ■  1 6.1 1  ♦  6.2 

□ 

7.0  Problem  Reports  Resolved • C8) 

7.1 

Nraber  of  those  problem  reports  that 

r“~| 

have  bean  closed  (r 

■solved). 

_ 1 

7.2 

Nraber  of  problem  reports  related 

to  the  requirements 

that  have  been 

□ 

Score  •  1 7.1 1  ♦  1 7.2-1 

□ 

II.  METRIC  WRXSECnCN: 

.  Sun  of  Above  Scores 

Metric  Value  -  No.  oFIEta  Elements 

□ 

III.  EVALUATION  NORKSBCTIGN:  that  is  your  evaluation  of  the 

reviewed  products  based  on  the  data  eleeeots  above? 

Cl-10) _ 

.Cl  If  you  are  unable  to  evaluate) 

TRACEABILITY  CHECKLIST 

Fora  Code:  CoPEM.2 

LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SCURCE(S) : 

□  SYSTB4 

□  SUBSYSTEM 

□  NODULE 

NAME: 

I.  DATA  COLLECTION  WORKSECTION: 

1.0  Cross  Reference  (traceability) (2S) 

1.1  Is  there  a  matrix  relating  itemized 

requirements  to  modules  vdiich  in-  —  — 

plement  these  requirements.  Y  N 

Yes  -  1,  No  -  0 

SCORE 

□ 

II.  METRIC  WORKSECTICN: 

.  Sum  of  Above  Scores 

Metric  Value  ■  Kb."  of  Data  Elements 

□ 

III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 

(1-10)  (jj  if  you  are  unable  to  evaluate) 

IV.  INSPECTOR’S  COMENTS:  ~~~! 

PREPARED  BY: _  APPROVED  BY: 


DATE: _  DATE: 


Co-22 


□□ 


DATA  CONSISTENCY  MEASURE 


Foxa  Code:  CoPEM.3 


1.0  Global  Date  (42) 

1.1  On  this  level,  is  global  data 
defined  only  once? 

Yes  •  1,  No  ■  0 


2.1  Is  standard  design  representation 
for  data  usage  established? 

Yes  -  1,  No  -  0 


II.  METRIC  WORKSECHCN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  MORBgCBCN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10)  (jj  If  you  are  unable  to  evaluate) 


CORRECTNESS 


Complete  this  scon  chart  if  an  application  of  this  set  of  worksheets  is 
appropriate  when  products  of  B3  sad  C  in  Analysis  Phase  became  available. 


Data  Answer  of  Data 

Element  Data  Flemrrr  Metric  Criteria  Factor 

Number  Element  Score  Scon  Score  Score 


Mater 


CORRSCDCSS 


Coaplete  this  score  chart  if  application  of  this  set  of  worksheet  is  appro¬ 
priate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration 
Phase  axe  available. 


Data  Answer  of  Data 

B«w nt  Data  ELenant  Metric  Criteria  Factor 


Data 


Nitric 


Criteria 


Factor 


PROCEDURE  CONSISTENCY  MEASURE 


LIFE  cm  PHASE: 

DETAIL  DI 

■SIQi 

O  SYSTBi 

0  SUBSYSTEM 
□  M3XJLE 

IJI1C  • 

Fona  Code:  CoDDM.l 


SOOO(S): 


I.  DATA.  COLLECTION  TCRKSECTION: 


II. 


III. 


1.0  Standard  Design  Representation  (101) 

1.1  Does  the  design  representation 
cosily  with  established  standards? 

Yes  ■  1,  No  ■  0 

2.0  Input/Output  Conventions  (102) 

2.1  Do  input/output  references  co^>ly 
with  established  standards? 

Yes  •  1,  No  •  0 

3.0  Calling  Sequence  Conventions  (103) 

3.1  Do  calling  sequences  asqply  with 
established  standards? 

Yes  •  1,  No  •  0 

4.0  Error  Handling  Conventions  (104) 

4.1  Is  error  handling  done  according 
to  established  standards? 

Yes  -  1,  No  -  0 


N 


N 


N 


METRIC  NORXSECnOK: 


Metric  Value 


Sum  of  Above  Scores 
'No.  of  Data  Elements 


EVALUATION  WRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  Eised  on  the  data  elements  above? 
(1-10) _ (j?  If  you  are  unable  to  evaluate) 


INSPECTOR'S  CEMENTS: 


□□ 


□□ 


COMPLETENESS  CHECKLIST 


Form  Code:  CoDEM.3 


G 


PREPARED  BY: 


APPROVED  BY: 


□□ 


CONSISTENCY 


Form  Code:  CoDDC.l 


CYCLE  PHASE: 

DETAIL  DESIGN 


SQQRCSCS): 

CoDEM.l 


METRIC  SUMMARY  TORXSECTION 


.  Procedure  Consistency  Measure 
2.  Data  Consistency  Measure 


CRITERIA  WDRXSECnON: 


...  Sun  of  Above  Score 
Criteria  Value  -  No.  of  Metrics - 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVE)  BY: 


COMPLETENESS 

Form  Code:  CoDDC.2 

LIRE  CYCLE  PHASE: 

DETAIL  DESIST 

saaocs): 

CoDCM.3 

□□□ 

I.  METRIC  SUMMARY  TOUSECTICN: 

1.  Completeness  Checklist 

SCORE 

□ 

•II.  CRITERIA  WRJ5ECTI0N: 

_ _  .  „  .  Sum  of  Above  Scores 

Catena  Value  Tfc-."  o?  Metrics - 

a 

III.  EVALUATION  WORKSECTTCN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 
f 1-103  (fi  If  you  are  unable  to  evaluate) 

APPROVED  BY: 


PREPARED  BY: 
DAIS: _ 


DATE: 


□□ 


QORRECPffiSS 


Complete  this  scon  char t  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  whan  products  of  B-D3  and  B-D4  is  Test  and  Integration  Phase  are 
available. 


Bata  Answer  of  Bata 
Element  Bata  Element 

Umber  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


TRACEABILITY 


Font  Code:  CoIMC.l 


CYCLE  PHASE: 

IMPLEMENTATION 


S0WQ5(S): 

CoMM.l 


□  suBsisax 

□  M3JQLE 


ICTRIC  SUMMARY 


Traceability  Checklist 


II.  CRITERIA  MORCECnON: 


......  Sum  of  Above  Scores  I  1 

Criteria  Value  «&;  0f  Metrics -  I - 1 


III.  EVALUATICN  WCRKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  eetrics  above? 

(1-10) _ (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 
DATE: _ 


Co-43 


CORRECTNESS 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

RELIABILITY  MODULE 

INTRODUCTION  TO  RELIABILITY 

The  Quality  Factor  Reliability  represents  a  measure  of 
the  extent  to  which  a  program  can  be  expected  to  perform  its 
intended  function  with  required  precision.  FIGURES  Re  -  1 
through  Re  -  4  represent  the  hierarchy  of  Metric,  Criteria 
and  Quality  Factor  worksheets  associated  with  Reliability  in 
Requirements  Analysis,  Preliminary  Design,  Detailed  Design 
and  Implementation  phases  of  the  Software  Development  Life 
Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS  * 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met- 
>  ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS:  t 

One  or  more  charts  tjiai  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  rpplied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Reliability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


CRITERIA 


METRIC 


FIGURE  Re-1 

REQUIREMENTS  ANALYSIS  WORKSHEET  HIERARCHY 


CRITERIA 


>C7RIC 


FIGURE  Re-2 

PRELIMINARY  DESIGN  WORKSHEET  HIERARCHY 


Re -3 


RELIABILITY 


QUALITY  FACTOR 


Re-4 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block-Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Re-5 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(S.l)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


Re  *6 


(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


Re-8 


HOW  TO  USE  SCORE  CHART  SETS 

When  you  apply  worksheets  to  system/subsystem/module , 
yon  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  Jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 
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Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart’s  Criteria 
Score  Column. 


Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


TRADE¬ 

OFF 

STUDY 


INITIAL  OR 
AUTHENTICATED 
SYSTBI 
SPEC, 


SET 
TO  BE 
APPLIH) 


'  ESD  PRODUCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  4  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  ICD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  -  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 


FUNC-  DE- 

TICNAL  TAILED 
PRODUCTS  FLOW  FLOW 

CHARTS  (HART 


SET 
TO  BE 
APPLIED 


DATA  ALGO- 
FORMAT  |  RI1H4 
DESC 


SYSTBf 

CPCI 
TEST 

TEST  |  PRO¬ 
PLANS 


ESD  PRODUCTS  FOR  EES  I  O'  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 

e  Product  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

e  Product  B.  Preliminary  Product  Specifications, 

including  the  above 

e  Product  D.  :  CDR  minutes  ft  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note, 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  information 

requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


PRODUCTS 


CODE 


ALTERED  PRODUCT  SPECS 


J 

WORKSHEET 

SET  TO  BE 

APPLIED 

DETAILED  DESIGN 

WORKSHEETS  (UPDATE) 

ESP  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

e  Product  C.  *  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor -supplied  information  should  be  searched. 


FINAL 

TO 


ING 

MODIFIED 

Him 

PRODUCT 

LEVEL 

SET  TO  BE 
APPLIED 


(UPDATE) 


ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system-level 
Test  Procedures 

B-D 

e  Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  Product  2:  Computer  Program  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

ICO  and  changea. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


Re -15 


RECOVERY  FRCM  COMPUTATIONAL  FAILURES  CHECKLIST 

Form  Code:  ReRAM*  2 


LIFE  CYCLE  PHASE: 

REQUIREMENT  ANALYSIS 


SYSTEM 


(SOURCE CS) : 


i  »  M 


HATA  CTLLECnCN  vnpiCTrrmN: 


tianal  Failures  (12) 


1.1  Are  there  definitive  statements  of 
the  requirements  for  recovery  from 
congutatianal  failures? 

Yes  -  1,  No  -  0 


II.  METRIC  WORKSECTTCN: 


III.  EVALUATION  W3RKSEQT0N:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


Metric  Value 


Sum  of  Above  Scores 


ESIZSi 


BY 


Re RAM. 3 


RECOVERY  FRCM  HARDWARE  FAULTS  CHECKLIST 

Fora  Code: 


LIFE  CYCLE  PHASE: 

REQUIREMENTS  ANALYSIS 


|J  SYSTB4 

□  SUBSYSTEM 

□  M3XJLE 


I.  DATA  OOLLECnON  WORKSECTION: 


I  SOURCE (S): 


1.0  Statements  of 


from  Hardware  Faults  (13) 


1.1  Is  there  a  definitive  statement  of 
the  requirement  for  recover/  from 
hardware  faults? 

Yes  •  1,  No  ■  0 


METRIC  WORXSECTICN: 


Metric  Value  ■ 


Sum  of  Above  Scores 


III.  EVALUATION  WDRKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (fi  If  you  are  unable  to  evaluate) 


BY 


BY 


RECOVERY  FRGM  DEVICE  ERRORS  CHECKLIST 

Fora  Code:  Re  RAM.  4 


LIFE  CYCLE  PHASE:  SOURCE (S) 

REQUIRB4ENT  ANALYSIS 


1.0  Statements  of 


from  Device  Errors  (14) 


1.1  Is  there  a  definitive  statement  of 
the  requirement  for  recovery  from 
device  errors? 

Yes  •  1,  No  •  0 


II.  METRIC  WQRXSECTICN: 


Metric  Value  ■ 


Sum  of  Above  Scores 


III.  EVALUATION 


(1-10) 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(JJ  If  you  are  unable  to  evaluate) 


ACCURACY  CHECKLIST 


Fora  Code:  Re  RAM.  5 


JFE  CIU£  PHASE: 

REQUIRBffiNT  ANALYSIS 

□  SYSTB4  |~N»g: 

□  SDBSYSTBi  - 

□  MODULE  - 


1.0  Error  Analysis  (9) 


SOUO(S): 


1.1  Has  an  error,  analysis  been  performed 
and  budgeted  to  function? 

Yes  -  1,  No  ■  0 

Accuracy  Requirement  (10) 

2.1  Are  there  definitive  statements 

of  the  accuracy  requirements  for  inputs, 
processing,  and  constants? 

Yes  ■  1,  No  ■  0 


II.  >CIRIC  WORXSBCnCN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  HOMSBCTION:  What  is  your  evaluation  of  the 

reviewed  products  based  on  the  data  elements  above? 
(1-10)  (3  If  you  are  unable  to  evaluate) 

IV.  INSPECTOR'S  GOfrBTCS:  ' 


PREPARED  3Y:a 
DATS: _ 


APPROVED  BY: 
OOE: _ 


SCORE 


Re -20 


i 


□□ 


ERROR  TOLERANCE 


Form  Code:  ReRAC.l 


liAly- 


Cm  FHASE: 

RBQUIRBENT  ANALYSIS 


SYSTEM 


SOURCE (S) : 

Re RAM. 1 

Re RAM. 2 

Re RAM. 3 

Re RAM. 4 ' 

METRIC  SlFWARY 


1.  Recovery  from  laproper  Input  Date  Checklist 

2.  Recovery  from  Caaputetioael  Failure  Checklist 

3.  Recovery  from  Hardware  Faults  Owcklist 

4.  Recovery  from  Device  Error  Checklist 


OOTERIA  WDRISECTION: 


.  Sum  of  Above  Score 

Criteria  Value  -  No.  o?  Metrics - 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(ft  If  you  are  unable  to  evaluate) 


-2 


□□□□ 


ACCURACY 

Font  Code:  ReRAC.2 

CYCLE  PHASE: 

REQUIRB4ENT  ANALYSIS 


METRIC  SUMMARY  VCRKSECTICN 


Accuracy  Checklist 


SOURCE(S): 
Re RAM. 5 


□ 


II.  CRITERIA  HCWSECnCW: 


Criteria  Value  ■ 


Sun  of  Above  Score 


III.  EVALUATION  VORKSECTIGN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (I  If  you  are  unable  to  evaluate) ' 


RELIABILITY 


Post  :  ReRAF .  1 


CYCLE  PHASE: 

REQUIREMENT  ANALYSIS 


SCDRSCS): 

ReRAC.l 


ReRAC.2 


I.  CRITERIA  SIMMARY 


1.  Error  Tolerance 


2.  Accuracy’ 


n.  factor  wgarngw: 


Factor  Value  » 


Sun  of  Above  Scares 


III.  EVALUATICN 


(1-10). 


Mint  is  your  evaluation  of  the 
on  the  criteria  above? 

(0  If  you  are  unable  te  evaluate) 


APPROVED  BY: 


D MI: 


Re -23 


RELIABILITY 


Conglete  this  score  chart  if  an  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  Al,  A2,  SI  and  B2  been—  available. 


FIRST  MEASUREMBfr  Date, 


Coaplete  this  score  chart  if  a  second  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2,  B1  and  B2  are  available. 

Data  Answer  of  Data 

h t  Data  El— nos  Metric  Criteria  Factor 

Naaber  ELeaent  Scots  Score  Score  Score 


ERROR  TOLERANCE  CONTROL  CHECKLIST 


Fora  Code:  RePIM.l 


(LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


|  Q  SYSTEM 
SUBSYSTB4 


UiUB  • 

jwns«a 


SOURCE (S) : 


i.  \fTf\  moRJCSEcncN: 

1.0  Centre!  Control  (31) 

1.1  Is  concurrent  processing  centrally 
controlled? 

Yes  •  1,  No  ■  0 

2.0  Error  Placable  end  Processing  Continued  (32) 

2.1  Nueber  of  errors  that  are  eutonati- 
cally  bypassed  tdtile  processing 
continues. 

2.2  Ntefcer  of  error  conditions  are  re¬ 
ported  by  the  systen. 


N 


I  I 

□ 


Score  ■ 


m  ♦ 


2.2 


II. 


III. 


IV. 


KCIRIC  jggBgCnCN: 

Metric  Value 


Sub  of  Above  Scores 
No.  of  Seta  Elements 


EVALUATION  WOREBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  Sued  on  the  data  elements  above? 
(1-10) _ {f  If  you  are  unable  to  evaluate) 


.INSPECTOR'S  oatgns: 


□□ 


□□ 


RECOVERY  FROM  DEVICE  ERROR  CHECKLIST 

Pom  Code:  RePEM.3 


III.  EVALUATION  WOMCSBCTICN:  What  is  you r  evaluation  of  the 
reviewed  products  oSsSi  on  the  data  eleaeots  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


ERROR  TOLERANCE 


Form  Code:  RePDC.l 


CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SYSTEM 


SOURCE (S) : 

RePEM.l 

RePEM.2 

RePEM.3 

I.  METRIC  SUMMARY  WORKSEOTCN: 


1.  Error  Tolerance  Control  Checklist 

2.  Recovery  from  Hardware  Faults  Qiecklist 

3.  Recovery  from  Device  Error  Checklist 


II.  CRITERIA  MORKSECTION: 


Criteria  Value  » 


Sum  of  Above  Score 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  aetrics  above? 

(0  If  you  are  unable  to  evaluate) 


□□□ 


SIMPLICITY 


Foot  Code:  RePDC.2 


METRIC  SUNMAKY 


1.  Design  Structure  Measure 


II.  CRITERIA  WORKSECTION: 


III.  EVALUATION 
(1-10) _ 


_  .  Sum  of  Above  Score 

Ciltera*  Value  -  No.' of" Retries - 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

0?  If  you  are  unable  to  evaluate) 


-3 


lL 


RELIABILITY 

Form  Code:_ 

Re PDF. 1 

LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SOURffi(S) : 

RePDC . 1 

RePDC. 2 

CRITERIA  SUMMARY  WCRXSECTION: 


1.  Error  Tolerance 

2.  Simplicity 


II.  FACTOR  WORKSECTION: 


_  .  ..  .  Sum  of  Above  Scores 

Factor  Value  -  fo/Vf  Criteria - 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  (JJ  If  you  are  unable  to  evaluate) 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 
DATE: _ 


Re-31 


RELIABILITY 


Complete  this  score  chart  if  an  application  of  this  set  of  worksheets  is  appro 
priate  when  products  of  35  and  C  in  Analysis  Phase  become  available. 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


FIRST  MEASUREMENT  Date _ 

Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  for  update  when  products  of  Cl,  C2  in  Design  Phase  are  available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


RELIABILITY 

Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appropriate 
for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration  Phase  are 
available. 


Data 

Element 

Number 


Answer  of 
Data 
Element 


Data 

Element 

Score 


Metric 

Score 


Criteria 

Score 


Factor 

Score 


THIRD  .MEASUREMENT  Cate _ 

Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration 
Phase  are  available. 


Data  Answer  of  Data 
Element  Data  Element 

Number  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


□o 


SYSTEM 


,1.0  Detected  Error  Condition  (72) 

1.1  When  an  error  condition  is  detected, 
is  it  passed  to  the  calling  nodule? 

Yes  -  1,  No  -  0 


II.  METRIC  WORKSECnON: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


^  s' 


-  --  -  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10)  (0  If  you  are  unable  to  evaluate) 


□□ 


DO 


ini;: 


RECOVERY  FRCM  COMPUTATIONAL  FAILURES  CHECKLIST 


Fora  Cods:  Re  DEM.  3 


CYCLE  PHASE:  S0UW2CS) 

DETAIL  DESIGN 


SYSTEM  *WC: _ 


ange  Test  for 
araoeters  7751 


and  Miltiple 


1.1  Are  loop  and  mltiple  transfer  index 
parameters  range  tested  before  use? 

Yes  -  1,  No  •  0 

Subscript  Checking  (79) 

2.1  Are  subscript  values  range  tested 
before  use? 

Yes  -  1,  No  -  0 


3.1  Is  output  checked  for  reasonableness 
before  processing  continues? 

Yes  ■  1,  No  ■  0 


II.  METRIC  WGRXSECnON: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


BY: 


COMPLEXITY  MEASURE 


Fora  Coda  :  & 


■  CYCLE  PHASE: 

DETAIL  DESIQ4 


SYSTB4 


MODULE 


Ui)A. 

IWS* 


SOURffl(S): 


DATA  COLLECTION  lCWSECTICH: 


1.0  Data  and  Control  Flow  Cowplexlty  (81) 

1.1  What  is  the  sub  of  the  nuBber  of 
decision  points,  subdecision  points, 
conditional  branches  and  uncondi¬ 
tional  branches? 


Score  «  "!  ■*• 


METRIC  WOBKSBCTICN: 


Metric  Value  ■ 


Sun  of  Above  Scores 
No.  of  Data  Elements 


EVALUATION  MOMSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elaaeats  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


T 


DESIGN  STRUCTURE  MEASURE 


Pom  Code:  ReD°^-6 


CYCLE  PHASE: 

DETAIL  DESIGN 


SOURCE (S) 


1.0  Module  Processing  not 


on  Prior 


1.1  Is  the  nodule  independent  of  knowledge  rn 

of  prior  processing?  IyIn! 

Yes  -  1,  No  -  0 

2.0  Entrance  end  Exit  of  the  Module  (83) 

2.1 

2.2  Nufcer  of  Exits  fro*  nodule.  _  I _ I 


Sow -pe.  ^20)  ♦  Jjl  ♦  [1  ♦  [LLUJ  ♦  ij 
Description  of  Input.  Output  .processing  end 

^ -  -  - 

3.1  Does  each  nodule  description  include  _ 

input,  output,  processing  and  lisdtations?  ( Y||N  I 

Yes  •  1,  No  -  0 


II.  METRIC  jCMBBCnCN: 


Metric  Value 


Sun  of  Above  Scores  |  H 

■  i e.  - J  miT'wrJS.  '——I 


Re -39 


□□ 


ERROR  TOLERANCE 


Font  Code: 


CYCLE  PHASE: 

DETAIL  DESIGN 


METRIC  SUMMARY 


1.  Error  Tolerance  Control  Checklist 

2.  Recovery  from  Improper  Input  Data  Checklist 

3.  Recovery  from  Computational  Failure  Checklist 


II.  OUTERIA  WORKSECnON: 


Criteria  Value 


Sun  of  Above  Score 


III.  EVALUATION  WCRXSBCTICN:  What  ia  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  0J  If  you  are  unable  to  evaluate) 


R-rRfTTT-g 


□□□ 


ACCURACY 


II.  CRITERIA  WCRXSBCTION: 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTICW:  Khat  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (9  If  you  are  unable  to  evaluate) 


□□ 


CYCLE  PHASE: 

DETAIL  DESIGN 


I.  METRIC  SIM4ARY  W3RXSECHCN 


1.  Complexity  Measure 

2.  Design  Structure  Measure 


II.  CRITERIA  WOWCSBCnON: 


Criteria  Value 


Sum  of  Above  Score 
no.  of  Metrics 


III.  EVALUATION  WURKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (I  If  you  are  unable  to  evaluate) 


□□□ 


RELIABILITY 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro 
priate  when  products  of  Al,  A2,  A3,  A4 ,  B  and  D  in  Design  Phase  became 
available. 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Fact 


AO-A 1  ?0  376  SYSTEMS  ARCHITECTS  INC  MI0OUN  NASI 

computer  Systems  ACQUISITION  NETMCS 

MAT  82 

UNCLASSIFIED  UBi 


f/§  t/t 

«•  vtuMt  it.  mmli—ctciu) 
.  HMMK4H7 
>IUI»  NL 


□□ 


RECOVERY  FRCM  IMPROPER  INPUT  DATA  CHECKLIST 


Fora  Code: 


LIFE  CYCLE  PHASE: 

IMPL04ENTATIGN 


SYSTB*  NAM2: 


{SOURCE  (S): 


1.1  Are  values  of  inputs  range  tested? 
Yes  -  1,  No  •  0 

2.0  Redundant  Input  Data  (75) 

2.1  Are  conflicting  requests  and  illegal 


Yes  -  1,  No  •  0 


3.1  Is  there  a  check  to  see  if  all 
necessary  data  is  available  before 
processing  begins? 

Yes  »  1,  No  •  0 

Input  Checking  (77) 

4.1  Is  all  input  checked,  reporting  all 
errors,  before  processing  begins? 

Yes  •  1,  No  ■  0 


II.  METRIC  WORJEECITCN: 


Metric  Value  ■ 


Sun  of  Above  Scores 


III.  EVALUATION  WQRKSECTION:  feat  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  eleaunts  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


RECOVERY  FRCM  COMPUTATIONAL  FAILURES  CHECKLIST 


Fotb  Code;  R®IM4.3 


CYCLE  PHASE:  SOURCE (S) 

IMPLBIENTATION 


1.1  Are  loop  and  multiple  transfer  index 
parameters  range  tested  before  use? 

Yes  ■  1,  No  •  0 

Subscript  Checking  (79) 

2.1  Are  subscript  values  range  tested 
before  use? 

Yes  ■  1,  No  »  0 

Output  Checking  (80) 

3.1  Is  output  checked  for  reasonable¬ 
ness  before  processing  continues? 

Yes  ».l.  No  "  0 


II.  1CIRIC  WGRXSECTIGN: 


Metric  Value  ■ 


Sun  of  Above  Scores 


III.  EVALUATION  MORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


I 


~  □□ 


□□ 


CODING  SIMPLICITY  MEASURE  (Continued) 

Form  Code:  ReINM.7 

LIFE  CYCLE  PHASE: 

IMPLEMENTATION 

SOURCE  (S) : 

□  SYSIB4 

1  |  SUBSYSTEM 

□  M2JUIE 

NAME: 

T.  DATA  COLLECTION  WORKSECTION  (continued): 

5.0  Unconditional  Branches  (111) 

5.1  Nunber  of  unconditional  branches.  M 

5.2  Nunber  of  executable  statements.  1  1 

Score  -  1  -  (l5.ll  ls.2l ) 

6.0  In  and  Out  of  Loops  (112) 

6.1  Nunber  of  one  entrance/  , - 1 

one  exit  loops.  II 

6.2  Total  Nunber  of  loops.  1  1 

Score  ■  6.1  -t-  6.2 

7.0  Loop  Index  (113) 

7.1  Nunber  of  loop  indices  that  are  i - i 

modified.  I_J 

7.2  Total  nunber  of  loops.  1  1 

Score  ■  1  -  (1 7,l|  4-  7.2  ) 

SCORE 

□ 

□ 

□ 

PREPARED  BY:  APPROVED  BY: 

DATE: _ _ _  DATE: _ 


Re -54 


□□ 


□□ 


ERROR  TOLERANCE 


Fora  Code i  ReIMC.1 


CYCLE  PHASE: 

IMPLEMENTATION 


NAJ€: 


SOURCE  (S) : 

Re  DM.1 

ReIMM.2 

Re  DM.3 


METRIC  SUMMARY 

1.  Error  Tolerance  Checklist 

2.  Recovery  from  Improper  Input  Data  Checklist 

3.  Recovery  from  Computational  Failures  Checklist 


CRITERIA  VCRX5ECTI0N: 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(0  If  you  are  unable  to  evaluate) 


BY 

APPROVED 

BY 

DATE: 

-56 


□□□ 


□□ 


ACCURACY 


Form  Code:  ReIMC.2 


CYCLE  PHASE: 

IMPLEMENTATION 


SOURCE (S) : 

ReINW-4 


METRIC  SUMMARY  WORJ5ECTION: 


1.  Accuracy  Checklist 


II.  CRITERIA  WORKSBCTION: 


.  .  Sum  of  Above  Scores  |  | 

Criteria  Value  «#,.■  0f  Metrics -  ‘ - - 


III.  EVAUIA3TC N  WORKSECTTON:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (JJ  If  you  are  unable  to  evaluate) 


REPARED  BY:, 
DATE: _ 


APPROVED  BY: 
DATE: 
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SIMPLICITY 


Form  Code:  ReDC-3 


CYCLE  PHASE: 

IMPLBENTATICN 


SOURCE(S) : 

Re  DM.  5 

Re  DM.6 

ReDM.7 


METRIC  SUMMARY 


1.  Design  Structure  Measure 

2.  Complexity  Measure 

3.  Coding  Simplicity  Measure 


II.  CRITERIA  WCRKSBCTICN: 


-  ....  _  Sum  of  Above  Score 

Cxiteri*  Value  -  ft,;  dTMetrxcs - 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(0  If  you  are  unable  to  evaluate) 


BY 


□□□ 


RELIABILITY 


Fora  Code:  ReIMF*1 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


SOURCE  (S) : 

ReIMC.1 


Re  DC.  2 


CRITERIA  SUMMARY  WCWCSECTICN 


1.  Error  Tolerance 

2.  Accuracy 

3.  Simplicity 


II.  FACTOR  WORKSECnON: 


Factor  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  an  the  criteria  above? 

(1-10) _  (jJ  If  you  are  unable  to  evaluate) 


□□□ 


a 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 
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INTRODUCTION  AND  INSTRUCTIONS 
FOR 

EFFICIENCY  MODULE 

INTRODUCTION  TO  EFFICIENCY 

The  Quality  Factor  Efficiency  represents  a  measure  of  the 
amount  of  computing  resources  and  code  required  by  a  program  to 
perform  a  function.  FIGURES  Ef  -  1  through  EF  -  4  represent  the1 
hierarchy  of  Metric,  Criteria  and  Quality  Factor  worksheets 
associated  with  Efficiency  in  Requirements  Analysis,  Preliminary 
Design,  Detailed  Design  and  Implementation  phases  of  the  Soft¬ 
ware  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con-  | 
sists  of  Metric  worksheets,  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle.  I 
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However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Efficiency  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


QUALITY  FACTOR 


CRITERIA 


tCIRIC 


FIGURE  Ef-1 

REQUIREMENTS  ANALYSIS  WORKSHEET  HIERARCHY 


FIGURE  Ef-2 

PRELIMINARY  DESIGN  WORKSHEET  HIERARCHY 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block -Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 
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Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions ,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element . 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  S  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value . 
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» 


(5.2) 

(5.3) 

(5.4) 

(5.5) 

(5.6) 

Step  6  - 
(6.1) 

(6.2) 

(6.3) 

(6.4) 
'  (6.5) 

(6.6) 


Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet . 

Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

FACTOR  WORKSHEETS 

Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

Answe’r  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 
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STEPS  FOR  UPDATE  APPLICATIONS 
Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 
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HOW  TO  USE  SCORE  CHART  SETS 

When  you  apply  worksheets  to  system/subsystem/module, 
yoa  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  £he  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  »  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 
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Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  ’’Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


L 
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Step  4  -  FACTOR  SCORE 


Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


TRADE 

OFF 

STUDY 


SET 
TO  BE 
APPLIED 


ESP  PRODUCTS  FOR  ANALYSIS  PHASE 

Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  ot  Authenticated  System  Specifi¬ 

cation  I  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  ICD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below'. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  -  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 


Ft NC- 
TICNAL 
PRODUCTS  FLOW 
CHARTS 


DATA 

ALGO- 

FORMAT 

RTITW 

rase. 

DESC. 

SET 
TO  BE 
APPLIED 


PRELIMINARY 

DESIGN 


ESP  PRODUCTS  FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 


Product  A.l:  Functional  flowcharts 
Product  A. 2:  Detailed  flowcharts 
Product  A. 3:  Data  format  descriptions 


Product  A. 4: 
Product  B. 
Product  0. 


Descriptions  of  algorithms  not 
previously  prescribed 

Preliminary  Product  Specifications, 
including  the  above 


e  Product  0.  CDR  minutes  6  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 


NOTE:  If  these  products  do  not  contain  the  information 

requested  by  the  Data  Element  quest \ons,  other 
contractor-supplied  information  should  be  searched. 
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ESP  PRODUCTS  FOR  POPING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  *  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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FINAL 

o>a 


SET  TO 
APPLIED 


ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPC1  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system- level 
Test  Procedures 

B-D 

e  “Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  “Product  2:  Computer  Program  Coding  Changes 

Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  “Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

ICS  and  changes.. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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sTiii 


EFFICIENCY 


Complete  this  score  chart  if  an  application  of  this  set  of  worksheet  is  appro 
priate  when  products  of  Al,  A2,  B1  and  B2  in  Analysis  phase  become  available: 


Data  Answer  of  Data 
Element  Data  Element 

Number  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


FIRST  >CASUREfrENT  Date 


Complete  this  score  chart  if  a  second  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2,  B1  and  B2  in  analysis  phase 
are  available: 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  See — 


STORAGE  EFFICIENCY  MEASURE 


Form  Code:  EfPOM.l 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SOURCE (S) : 

□□□ 

NAME: 

_ 1 

I.  DATA  OOLLECnON  TORKSECTION: 

1.0  Storage  Allocation  Requirements  (381 

1.1  Are  storage  requirements  alloca-  — i— i 

ted  to  design?  Y  N 

Yes  ■  1,  No  ■  0 

2.0  Use  of  Virtual  Storage  (391 

2.1  Are  virtual  storage  facilities  p- i — 

used?  Y  N 

Yes  -  1,  No  -  0 

□  □ 

II.  METRIC  WORXSECTION: 

.  Sum  of  Above  Scores 

Metric  Value  *  No.  of  Data  Elements 

a 

••III.  EVALUATION  WORXSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 

IV.  INSPECTOR'S  CCWMENTS: 

PREPARED  BY: 
DATE: _ 


APPROVED  BY:. 
DATE: _ 
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□□ 


ITERATIVE  PROCESSING  MEASURE 


Font  Coda 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIQJ 


UUJ}. 

□ - 1 

□ 


|SOOO(S} : 


1.1  Gb  this  level,  is  data 
packing  used? 

Yes  ■  1,  No  -  0 


NETRIC  NORXSECTICN: 


Sun  of  Above  Scores 
Mettle  Value  -  No.  of  Data  EXSSots 


III.  EVALUATION  WORKSECTICW:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (JJ  If  you  are  unable  to  evaluate) 


□□ 


□□ 


ills 


EFFICIENCY 


Fora  Code:  EfFDF.l 


EFFICIENCY 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  53,  and  C  in  analysis  phase  axe  available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


FIRST  MEASUREMENT  Date _ 

Complete  this  score  chart  if  application  of  this  set  of  Worksheets  is  appro¬ 
priate  for  update  when  products  of  Cl  and  C2  in  Design  Phase  are  available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  ScoTe  Score 


38 

39 

43 

44 


SECOND  MEASUREMENT 


Date 


EFFICIENCY 


SSr*^-“'as!  *w-  *  mss; 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


38 

39 
43 


44 


THIRD  FCASURBBff  Date 


Collet,  this  spot  chart  if  a  fourth  application  of  this  set  of  worksheet,  is 

Pr°to’  °f  “lKt  ■*  Intention 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score  , 


38 

39 

43 

44 


FOURTH  MEASUREMENT 


Date 


DATA  COLLECTION  VOUCSECTION: 


1.0  Data  Efficiency  (43) 

1.1  On  this  level,  is  the  data  in* 
dexed  or  referenced  efficiently? 

Yes  -  1,  No  •  0 


METRIC  WQRX5ECTI0N: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  VQRKSECTION;  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


PERFORMANCE 


IRB4ENTS  CHECKLIST 

Fora  Code:  EfDEM.2 


CYCLE  PHASE: 

DETAIL  DESIGN 


ATA  COLLECTION  UPKXSECHQN: 


1.1  Are  specific  performance  require¬ 
ments  allocated  to  this  module  in 
the  Design  Phase? 

Yes  -  1,  No  -  2 


METRIC  NORKSECHCN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  clamant?  above? 
(1-10) _  (3  If  you  are  unable  to  evaluate) 


□□ 


ITERATIVE  PROCESSING  MEASURE 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 


1.0  Non-Loop  Dependent  Computations  Kept  Out  c 

1.1  How  nosy  non- loop  functions  ire  kept 
cut  of  loops? 

1.2  Total  xuaber  of  loop  statements. 

Score  ■  EH  *  E2 
2.0  Data  Packing  (44) 

2.1  On  this  level,  is  bit/byte 

packing/unpacking  kept  out  of  loops? 


II.  METRIC  WORKSECnCN: 


...  .  Sum  of  Above  Scores 
Metric  Value  «  No.  0£  Data  Elements 


III;  EVALUATION 
(1-10) _ 


ON:  What  is  your  evaluation  of  the 
sed  fin  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 
DATE: _ 


APPROVED  BY:. 
DATE: _ 


Ef -30 


□□□ 


EFFICIENCY 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  C  in  Coding  and  Checkout  phase  is  available. 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


43 
26 
100 

44 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appropriate 
when  products  of  B-D3,  B-D4  in  Test  and  Integration  Phase  are  available. 


Answer  of  Data 

Data  Element 

Element  Score 


Metric 

Score 


Data 

Element 

NumbcT 


Criteria 

Scotc 


Factor 

Score 


DATA  COLLECTION  WORKSECTION: 

1.0  Date  Grouped  for  Efficient  Processing  (13 

1.1  Have  data  base  or  files  been 

organized  for  efficient  processing? 

Yes  -  1,  No  -  0 


METRIC  WORKSECTICN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECHCN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (jJ  If  you  are  unable  to  evaluate) 


□□ 


STORAGE  EFFICIENCY 


fora  Codg:  EfIMC.l 


□□ 


EFFICIENCY 


Fora  Code:  EfIMF.l 


SCURQ(S) : 

EfIMC.l 

EfIMC.2 

CRITERIA  SUMARY  VCRKSECTION: 


1.  Storage  Efficiency 

2.  Execution  Efficiency 


FACTOR  WQRXSECnON: 


Factor  Value 


Sum  of  Above  Scores 
No.  o£  Criteria 


III.  EVALUATION 
(1*10) _ 


PREPARED  BY: 

DATE: _ 


What  is  your  evaluation  of  the 
on  the  criteria  above? 

(j!  If  you  are  unable  to  evaluate) 


______________  APPROVED  BY: 

_  .  DATE: _ 


Ef-39 


EFFICIENCY 

Complete  this  score  chert  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  A  and  B  in  Coding  and  Checkout  Phase  are  available. 
Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro 
nri.ate  when  products  of  B-D2  in  Test  and  Integration  Phase  is  available. 
Data  Answer  of  Data 

Elenmnt  Data  Element  Metric  Criteria  Fact) 

Element  Score  Score  Score 


SEOOND  MEASUREMENT 


Goeplete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro  - 
priate  when  predicts  of  B-D2  in  Test  and  Integration  Phase  is  available. 

Data  Answar  of  Data 

Element  Data  Elmeaat  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


It-i 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

INTEGRITY  MODULE 

INTRODUCTION  TO  INTEGRITY 

The  Quality  Factor  Integrity  represents  a  measure  of  the 
extent  to  which  access  to  software  or  data  by  unauthorized  per¬ 
sons  can  be  controlled.  FIGURES  It  -  1  through  It  -  4  repre¬ 
sent  the  hierarchy  of  Metric,  Criteria  and  Quality  Factor  work¬ 
sheets  associated  with  Integrity  in  Requirements  Analysis,  Pre¬ 
liminary  Design,  Detailed  Design  and  Implementation  phases  of 
the  Software  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Integrity  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block- Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block-On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products. 
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Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  bloc  .. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0 . 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


It-S 


(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 
% 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  ’’Update  Information 
Requirements”,  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets”,  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 
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HOW  TO  USE  SCORE  CHART  SETS 


When  you  apply  worksheets  to  system/subsystem/module, 
you  must  apply  some  worksheets  more  than  once  in  the  Com* 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


PRODUCTS 


TRADE¬ 

OFF 

STUDY 


INITIAL  OR 
AUTHENTICATED 
SYSTS4 
SPEC 


I  AUTHENTICATED 

HIGHER 

DRAFT 

MINUTES 

'■"Tr'i 

SPEC 

FOR 

CPCI 

LEVEL 

SPEC 

PRODUCT 

SPECS 

ACTION 

ITB< 

RESPONSES 

SET 
TO  BE 


REQUIREMENTS 

ANALYSIS 


ESP  PRODUCTS  FOR  ANALYSIS  PHASE 

Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  f  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  ICD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 
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RJNC-  DE- 
TICNAL  TAILED 
PRODUCTS  FLOW  FLOW 

QUARTS  CHART 


DATA  ALGO- 
FORMAT  I  RITH4 
DESC. 


SYSTB4 

PRELIM. 

CPCI 

MENT 

TEST 

TEST 

PRO- 

PLANS 

SET 
TO  BE 
APPLIED 


ESD  PRODUCTS  FOR  DESIOJ  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

•  Product  A. 3:  Data  format  descriptions 


Product  A.l: 
Product  A. 2: 
Product  A. 3: 
Product  A.  4: 


•  Product  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

•  Product  B.  :  Preliminary  Product  Specifications, 

including  the  above 

e  Product  D.  :  CDR  minutes  4  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note, 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  information 

requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 
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PRODUCTS 


CODE 


ALTERED  PRODUCT  SPECS 


W0RX5HEET 
SET  TO  BE 
APPLIED 


nriaerrAiroN  detailed  desist 

WORKSHEET'S  WORKSHEETS  (UPDATE) 


ESD  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 


Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
»•  the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 


ESP  PRODUCTS  FOR  TEST  AND 


INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  systen>level 
Test  Procedures 

B-D 

e  “Product  Is  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  Product  2:  Computer  Program  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  “Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

ICS  and  changea. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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DO 


ACCESS  CONTROL  CHECKLIST 


Form  Code:  ItRAM.l 


CYCLE  PHASE: 

RBQUIRB4ENTS  ANALYSIS 


ISCURCECS) 


Input/Output  Access  Control  (15) 

1.1  Is  there  e  definitive  statement  of 
the  requirement  for  user  input/output 
access  controls  in  the  Requirements 
Analysis  Phase? 

Yes  -  1,  No  -  0 

Data  Base  Access  Control  (16) 

2.1  Is  there  a  definitive  statement  of 
the  requirement  for  data  base  access 
controls  in  the  Requirements  Analysis 
Phase? 

Yes  -  1,  No  •  0 
Memory  Protection  (17) 

3.1  Is  there  a  definitive  statement  of 
the  requirement  for  memory  protection 
across  tasks  in  the  Requirements  Anal¬ 
ysis  Ihase? 

Yes  ■  1,  No  -  0 


JCTRIC  NORJSECTICN: 


Metric  Value  • 


Sum  of  Above  Scores 


III.  EVALUATION  WQWSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


iiaiL 


ansi 


S 


ACCESS  AUDIT  CHECKLIST 

Fora  Code:  ItRAM.2 

LIFE  CYCLE  PHASE: 

REQUIREMENT  ANALYSIS 

SOURCE  (S): 

□□□ 

NAME: 

I.  DATA  COLLECTION  RCRJCSECnCN: 

SCORE 

1.0  Recording  and  Reporting  Access  CIS-) 

1.1  Are  there  definitive  statements 

of  the  requirements  for  recording  and 
reporting  access  in  the  Requirements  r— i— , 

Analysis  Phase?  Y  N 

Yes  -  1,  No  -  0 

□ 

2.0  Indication  of  Access  Violation  (191 

2.1  Is  there  a  definitive  statement  of 
the  requirements  for  immediate  indi- 
cation  of  access  violation  in  the  — j— 

Requirements  Analysis  Phase?  Y  N 

Yes  «  1,  No  -  0 

□ 

II.  METRIC  WORXSECTIGN: 

.  Sun  of  Above  Score 

Metric  Value  «  No.  of  Data  Elements 

□ 

III.  EVALUATION  WCRKSBCTICN:  What  is  vour  evaluation  of  the 
reviewed  products  basSi  on  the  data  elements  above? 

(1-10)  (9  If  you  *re  unable  to  evaluate) 

IV.  INSPECTOR'S  OMENTS: 

PREPARED  BY: _  APPROVED  BY: 


DAZE; _  DA3E: 
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ACCESS  CONTROL 


Fora  Code:  ItRAC.l 


CYCLE  PHASE:  SOURCE (S) : 

REQUIRB4ENTS  ANALYSIS  ItRAM.l 


NETRIC  SUMMARY  WORKSECTION 


1.  Access  Control  Checklist 


II.  CRITERIA  NORXSECnON: 


Criteria  Value 


Sum  of  Above  Score 


III.  EVALUATICN  WOfUSECTION:  What  is  your  evaluation  of  the 
reviewed  products  tSSSd  an  the  netrics  above? 

(1-10) _  &  If  you  are  unable  to  evaluate) 


I 


ACCESS  AUDIT 


Form  Code:  ItRAC.2 


CYCLE  PHASE: 

REQUIREMENTS  ANALYSIS 


SCURCE(S): 
ItRAM.  2 


I.  METRIC  SLFWARY  VCRKSECTION 
1.  Access  Audit  Checklist 


II.  CRITERIA  WCMSECTION: 


Sum  of  Above  Score 


Criteria  Value  - 


III.  EVALUATION  WORKSBCTION:  What  is  your  evaluation  of  the 
reviewed  products  Based  on  the  metrics  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: 


APPROVED  BY: 

m 

OATS: _ 
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INTEGRITY 


LITE  CYCLE  PHASE:  SOURGE(S): 

REQUIREMENTS  ANALYSIS  ItRAC.l 

H  SYSTEM 


Form  Code:  ItRAF.l 


ItRAC.2 


X.  Access  Control 
2.  Access  Audit 


H.  FACTOR  RORKSECTION: 


Factor  Value  -  Sum  of  Above_ScoTes 
No.  of  Criteria 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


PREPARED  EY: 
DATE: _ 


APPROVED  BY: 
DATE: _ 
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INTEGRITY 


Complete  this  score  chart  if  an  application  of  this  set  of  worksheets  is  appro 
priate  when  products  of  Al,  A 2,  £1  and  £2  in  Analysis  Phase  beccraes  avails' 
ble. 

Data  Answer  of  Data 

ELeneaat  Data  Element  Metric  Criteria  Factor 

Number  Elenent  Score  Score  Score  Score 


OD 


ACCESS  CONTROL  CHECKLIST 


ILTFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SY5TE4 
SUBSYSIEM 
MXQLE 


NAME:. 


SOURCE  (S3 : 


I. 


DATA  OOLLECTTOl  HXXSECTTaH: 


Form  Code:  ItPDM.l 


SCORE 


1.0  Inpot/Output  Access  Control  (IS) 

1.1  Are  user  input/output  access  controls 
provided  in  the  Design  and  Implementa¬ 
tion  Phases? 

Yes  -  1,  No  -  0 

2.0  Data  Base  Access  Control  (16) 

2.1  Are  data  base  access  controls  pro¬ 
vided  in  the  Design  and  Implementa¬ 
tion  Phases? 

Yes  ■  1,  No  ■  0 

3.0  Memory  Protection  (17) 


□ 

□ 


II. 


III. 


3.1  Is  memory  protection  across  tasks 
provided  in  the  Design  and  Iaple- 
mentation  Phases? 

Yes  ■  1,  No  ■  0 


METRIC  W3RKSECTI0N: 

Metric  Value 


Sum  of  Above  Scores 
No.  of  Data  Elements 


EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 


(1-10) _  (Jl  If  you  are  unable  to  evaluate) 


□o 


ACCESS  AUDIT  CHECKLIST 


Fora  Code:  ItpCM-2 


[LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SOURCE (S) : 


n  SYSTBi 
SUBSYSTBi 


NAME: 


M3XJLE 


I.  DATA  OOLLECTION  WRKSECTION: 

1.0  Recording  and  Reporting  Access  (18) 

1.1  Are  provisions  for  recording  and 
reporting  access  provided  in  the 

Design  and  Implementation  phases? 

Y  N 

Yes  -  1,  No  -  0 

□ 

2.0  Indication  of  Access  Violation  (19) 

2.1  Are  provisions  for  indication  of 
access  violation  provided  in  the 

Design  and  Implementation  Phases? 

Y  N 

□ 

Yes  •  1,  No  -  0 

II.  METRIC  WORXSECTICN: 

Sun  of  Above  Scores 
Metric  Value  ■  No.  of  Data  Elements 

□ 

III.  EVALUATION  WCRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (j?  If  you  are  unable  to  evaluate) 


IV.  INSPECTOR’S  COMMENTS: 


INTEGRITY 


Fora  Coda:  ItPDF.l 


SOURCE  (S) : 

ItPDC.l 

ItPDC.2 

I.  CRITERIA  SUMMARY  WCRJSECTION 

1.  Access  Control 

2.  Access  Audit 


II.  FACTOR  M3RKSECTICN: 


Factor  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  baled  on  the  criteria  above? 

(1-101  (jl  If  you  are  unable  to  evaluate) 


{REPARED  BY:. 
DATE: _ 


APPROVED  BY: 

m 

DATE: _ 
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INTEGRITY 


Complete  this  score  chart  if  an  application  of  this  set  of  worksheets  is 
appropriate  when  products  of  B3  and  C  in  Analysis  Phase  becoaie  available. 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Nunber  Element  Score  Score  Score  Score 


FIRST  MEASURBENT  DATE 


Complete  this  score  chart  if  a  second  application  of  this  set  of  worksheets 
is  appropriate  for  vpdate  when  products  of  Q  and  C2  in  Design  phase  become 
available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Actor 

Ntmber  Element  Score  Score  Score  Score 


INTEGRITY 


Coaplete  this  scots  chart  if  a  third  application  of  this  set  of  worksheets 
is  appropriate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Inte¬ 
gration  Phase  become  available. 


Number 


Answer  of 
Seta 

Element 


Score 


Metric 

Score 


Criteria 

Score 


Factor 

Score 


Complete  this  score  chart  if  a  fourth  application  of  this  set  of  worksheets 
is  appropriate  for  update  when  products  of  Al  and  A2  and  B-Dl  in  Test  and 
Integration  Phase  are  available. 


Data 

Element 


Answer  of 
Data 
Element 


Metric  Criteria  Factor 

Score  Score  Score 


F0UR3H 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 
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INTRODUCTION  AND  INSTRUCTIONS 
FOR 

USABILITY  MODULE 
INTRODUCTION  TO  USABILITY 

The  Quality  Factor  Usability  represents  a  measure  of  the 
effort  required  to  learn,  operate,  prepare  input,  and  interpret 
output  of  a  program.  FIGURES  Us  -  1  through  Us  -  4  represent 
the  hierarchy  of  Metric,  Criteria  and  Quality  Factor  worksheets 
associated  with  Usability  in  Requirements  Analysis,  Preliminary 
Design,  Detailed  Design  and  Implementation  phases  of  the  Soft¬ 
ware  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 
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However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  ependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  inst  -’actions  show  when  the  Usability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


STEPS  FOR  COMPLETING  WORKSHEETS 


Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block -Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  >aule  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 
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(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 


(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 
Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  requirei 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 
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HOW  TO  USE  SCORE  CHART  SETS 


When  you  apply  worksheets  to  system/ subsystem/module , 
you  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 
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Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


PRODUCTS 

TRADE¬ 

OFF 

STUDY 

REPORTS 

INITIAL  OR 
AUTHENTICATED 
SYSTB-f 

SPEC, 

SEGMENT 

SPEC 

AUTHENTICATED 

DEVELOPMENT 

SPEC 

FOR 

CPCI 

KCfflER 

LEVEL 

SPEC 

DRAFT 

PRODUCT 

SPECS 

PDR 

MINUTES 

AND 

ACTION 

ITEM 

RESPONSES 

WORKSHEET 

SET 

TO  BE 
APPLE® 

REQUIREMENTS 

ANALYSIS 

WORKSHEETS 

PRELIMINARY 

DESIGN 

WORKSHEETS 

ESP  PRODUCTS  FOR  ANALYSIS  PHASE 

Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  I  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  foT 

each  CPC I 

-  Product  B.2  -  Possible  higher-level  specification,  ICD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 
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X~** 


FUNC-  . 

DE¬ 

TIONAL 

TAILED 

PRODUCTS 

FLOW 

FLOW 

GiARTS 

CHART 

SET 
TC  BE 
APPLIED 


SYSTBf 

CPCI 
TEST 

TEST  |  PRO- 
PLANS 


ESD  PRODUCTS  FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 


Product  A.l: 
Product  A. 2: 
Product  A. 3: 
Product  A. 4: 

Product  B.  : 

Product  D .  : 


Functional  flowcharts 

Detailed  flowcharts 

Data  format  descriptions 

Descriptions  of  algorithms  not 
previously  prescribed 

Preliminary  Product  Specifications, 
including  the  above 


e  Product  D.  CDR  minutes  4  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


PRODUCTS 

CODE 

ALTERED  PRODUCT  SPECS 

S 

WORKSHEET 

SET  TO  BE 

APPLIED 

IMPLEMENTATION 

DETAILED  DESIGN 

WORKSHEETS 

WORKSHEETS  (UPDATE) 

ESP  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
>•  the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 


Us -13 


FINAL 

cpq 


SEGMENT  * 

st srm 


CODING  I  PRODUCT 


ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 


Product  1: 
Product  2: 


Final  CPCI  Test  Procedures 

Segment  (if  any)  and  system- level 
Test  Procedures 


e  Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

•  Product  2:  Computer  Program  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 


B-D 

Product  3j 


Modified  Product  Specification 


Product  4:  Possible  high-level  specification, 
I CD  and  changes. 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


□□ 


CYCLE  PHASE: 

REQUIRH*tENTS  ANALYSIS 


OPERABILITY  CHECKLIST 


SOURCE (S) 


Form  Code:  UsRAM.l 


Operating  Characteristics  (20) 

1.1  Are  all  steps  in  the  operation 
described? 

Yes  -  1,  No  -  0 

Error  Conditions  end  Responses  (21) 

2.1  Are  all  error  conditions  to  be 
reported  to  the  operator/user 
identified,  and  are  the  responses 
described? 

Yes  •  1,  No  ■  0 

Operator's  Capabilities  (22) 

3.1  Is  there  a  definitive  statement  of 
the  requirements  for  the  capability 
to  interrupt  operation,  obtain  sta¬ 
tus  ,  modify,  and  continue  processing 
in  the  Requirements  Analysis  Phase? 
Yes  ■  1,  No  ■  0 


II.  METRIC  WORKSECTICN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


7i  1  '■  r:  ^  im  ?  r>+. 


BY 

APPROVED 

BY 

DATE: 

□□ 


USER  OUTPUT  INTERFACE  MEASURE 


Form  Code:  UsRAM 


CYCLE  PHASE: 

REQUIREMENTS  ANALYSIS 


DATA  COLLECTION  WORKSECHON: 


1.1  Is  there  a  definitive  statement  of 
the  requirement  for  optional  output 
media  in  the  Requirements  Analysis 
Phase? 

Yes  -  1,  No  «  0 
2.0  Output  Control  (25) 

2.1  Is  there  a  definitive  statement  of 
the  requirements  for  selective  output 
control  in  the  Requirements  Analysis 
Phase? 

Yes  -  1,  No  •  0 


METRIC  WORKSECTION: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


TRAINING  CHECKLIST 


SCURCE(S): 


Fora  Code: 


UsRAN.4 


DATA  COLLECTION  WORXSECTION: 

1-0  Documentation  of  Lesson  Plena /Training  Material  (54) 


1.1  Are  lesson  plans/training  materials 
provided  for  operators,  users,  and 
maintainors? 

Yes  -  1,  No  -  0 

2.0  Realistic  Simulated  Exercises  (55) 


SCORE 


3.1  Are  "Help"  and  diagnostic  inform 
tian  provided? 

Yes  -  1,  No  •  0 


II. 

METRIC  WDRXSECITCN: 

□ 

_ _ i  _  v-i _ Sum  of  Above  Scores 

No.  ox  Data  Elements 

III. 

EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  an  the  data  elements  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 

IV. 

INSPECTOR'S  CCM4ENTS: 

BY 


BY 


OPERABILITY 


CYCLE  PHASE:  SOURCE (S): 

REQUIREMENTS  ANALYSIS  Us RAM. 1 


Fora  Code:  UsRAC.l 


□□ 


OCMUNICATIVENESS 


Pom  Code:  UsRAC.2 


CYCLE  PHASE: 

REQUIREMENT  ANALYSIS 


SYSTEM  me: _ 


SGURffi(S): 

UsRAM.2 


METRIC  SUMARY  WORKSECTION: 

1.  User  Output  Interface  Measure 

2.  User  Input  Interface  Measure 


CRITERIA  WDRKSECnON: 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECnON:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (jJ  If  you  are  unable  to  evaluate) 


USABILITY 


Fora  Code: 


LIFE  CYCLE  PHASE:  SOURCE (S)  : 

REQUIREMENT  ANALYSIS  UsRAC.l 


Us RAC. 2 


CRITERIA  SUMMARY  ICRXSECTION: 

1.  Operability 

2.  Conramicativeness 

3.  Training 


FACTOR  WORKSECnON: 


Factor  Value  -  jg"  g| 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  (jj  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


□□□ 


OPERABILITY  CHECKLIST 


Fora  Code:  UsPEM.l 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


DATA  COLLECTION  XORXSECHON: 

1.0  Operating  Characteristics  (20) 


1.1  Are  all  steps  in  the  operation 
described? 

Yes  -  1,  No  •  0 

2.0  Error  Conditions  and  Responses  (21) 

2.1  Are  all  error  conditions  to  be  re¬ 
ported  to  the  operator/useT  identi¬ 
fied,  and  axe  the  responses  de¬ 
scribed? 

Yes  •  1,  No  ■  0 


3.1  Are  provisions  for  the  operator  to 
interrupt  operation,  obtain  status, 
modify,  and  continue  processing  pro¬ 
vided  in  the  Design  Phase? 

Yes  ■  1,  No  -  0 


4.0  Hard 


Maintenance  (53) 


4.1  Is  a  hard  copy  of  operator  inter¬ 
actions  to  be  aaintained?  < 

Yes  •  1,  No  ■  0 


BY 


BY 


OPERABILITY  CHECKLIST  (Continued) 


Fom  Code:Usra,L1 


CYCLE  PHASE: 

PRELIMINARY  DESIGN 


DATA  COLLECTION 


5.1  Are  the  interactions  between  the 
operator  and  systems  simple  and 
consistent? 

Yes  ■  1,  No  ■  0 


II.  METRIC  NORKSECTION: 


III.  EVALUATION  WQRKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


Metric  Value 


Sum  of  Above  Scores 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY: 
DATE: _ 


Us -25 


Fora  Code:  UsPDM.2 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


(SOURCE (S): 


SYSTB4 


NAIC: 


DATA  COLLECTION  VORXSECTION: 


1.1  Are  provisions  for  input  from  different 
media  provided  in  the  design  phase? 

Yes  -  1,  No  -  0 

2.0  Uniform  Input  Formats  (57) 

2.1  Number  of  input  formats 
Score  »  1  +■  2.1 

3.0  Default  Values  (58) 

3.1  Number  of  default  values. 

3.2  Number  of  input  values. 

Score  ■  3.1  +  j  3.2  | 

4.0  Self- Identifying  Trmut<;  (59) 

4.1  Number  of  records  with  self-defining 
input  codes. 

4.2  Total  Number  of  input  records. 

Score  ■  U.l  I  +  4.2 


USER  INPUT  INTERFACE  MEASURE  (Continued) 

Form  Code:  UsPDM.2 


300105(5) 


DATA  COLLECTION  WRJSECTICN  f continued 


Verification  (60) 


5.1  Can  input  be  verified  by  the  , 
user  prior  to  execution? 

Yes  -  1,  No  •  0 
licitly  Defined  Logical  End  of 


6.1  Is  input  terminated  fay  explicitly 
defining  the  and  of  input? 


Yes  ■  1,  No  ■  0 


METRIC  WORKSECTION: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


OD 


USER  OUTPUT  INTERFACE  MEASURE 


Form  Code: 


UsPDM.3 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESI® 


[j  SYSIB4 
SUBSYSTEM 
MXULE 


me: 


SOURCE (S) : 


I. 


DATA  COLLECTION  WORKSECTION: 


SCORE 


1.0 


2.0 


3.0 


4.0 


Flexibility  of  Output  (24) 

1.1  Are  provisions  for  directing  output 
to  different  media  provided  in  the 
Design  and  Iaplementation  Phases? 

Yes  ■  1,  No  ■  0 

Output  Control  (25) 

2.1  Are  provisions  for  selective  output 
controls  provided  in  the  Design  and 
Implementation  Phases? 

Yes  -  1,  No  -  0 

Techniques  for  Separating  Logical  Groups 
of  Output  (62) 

3.1  Are  logical  groups  of  output  separ¬ 
ated  for  user  examination? 

Yes  -  1,  No  -  0 

Relationship  Between  Error  Messages  and 
Outputs '  (635 — * - “ - B - 

4.1  Are  the  relationships  between 
error  messages  and  outputs  unam¬ 
biguous? 

Yes  •  1,  No  ■  0 


USER  OUTPUT  INTERFACE  MEASURE  (Continued) 

Fom  Code:  UsPDM.3 


DATA  COLLECTION  WCRXSECTICN  (cant 


S.l  Do  outputs  have  descriptive 
user-oriented  labels? 


Yes  •  1,  No  ■  0 


6.0  Individual 


Item  0W2) 


6.1  Do  outputs  have  user  oriented  units?  |Y)N[ 
Yes  -  1,  No  -  0 

7.0  Uniform  Output  Formats  (143)  | _ ) 

7.1  Number  of  Output  formats?  I  1 

Score  •  1  «*  7.1 


II.  METRIC  WORKSBCTION: 


III.  EVALUATION  MORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  blsed  on  the  data  elements  above? 
(1-10) _ (jf  If  you  are  unable  to  evaluate) 


Metric  Value 


Sum  of  Above  Scores 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 

m 

DA2E: _ 


Us  -29 


OPERABILITY 


Font  Code: 


U3V»J 


LIFE  CYCLE  PHASE: 

SCURQ2CS): 

PRELIMINARY  DESIGJ 

UsPEM.l 

METRIC  SOMARY 


1.  Operability  Checklist 


CRITERIA  WDRJSECnON: 


_ r.  tr-i _ Sum  of  Above  Scores  I  I 

Criteria  Value  -  ^  gtHcs -  1 - 1 


HI.  EVALUATION  W0RKS5CTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


s  -  3 


amJNICATIVENESS 

Fora  Code:  UsPDC.Z 

LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SOURCE (S) : 

UsPDM.2  UsPDM.3 

□□□ 

NAftC: 

I.  METRIC  SUMMARY  WORXSECTION: 

1.  User  Input  Interface  Measure 

2.  User  Output  Interface  Measure 

SCORE 

□ 

□ 

II.  CRITERIA  WORKSECnON: 

Criteris 

.  _ Sum  of  Above  Score: 

□ 

i  value  jg,,  oi  Metrics 

III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 

PREPARED  BY: 


APPROVED  BY: 


USABILITY 


Form  Code: 


UsPDF.l 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SOURCE(S): 

UsFDC.l 

UsPDC.2 

CRITERIA  SUMMARY  VORKSECTION 
1.  Operability 


2.  Communicativeness 


II.  FACTOR  W3RKSECTI0N: 


Factor  Value 


Sum  of  Above  Scores 
No.  of  Criteria 


III.  EVALUATION  WORJCSECTION:  Whet  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  (j?  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY: 
DATE: _ 


Us-32 


USABILITY 


Conplete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  B 3  and  C  in  Analysis  Phase  are  available. 


Data  Answer  of  Data 


Data  n—wt  Metric  Criteria  Factor 

Eleaaant  Score  Score  Score  Score 


Us-  33 


USABILITY 


Complete  this  score  chart  if  a  second  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Cl  and  C2  in  Design  Phase  are  available 

Data  Answer  of  Data 

Fifwmt  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  ScoTe 


USABILITY 


Cocnlete  this  score  chart  if  a  third  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2,  and  B-Dl  in  Test  and  Integration 
rtiase  are  available. 


THIRD  MEASUREMENT  Date 


Us -35 


f!l  ^!I  4  4  1 4  4  i  N  4 


gFT.TARH.TTY 

i-w.  c  score  chart  if  application  of  *hi*  set  of  worksheets  is  appro- 
or  update  when  products  of  Al,  A2  sad  B-Dl  in  Test  and  Integration 
e  available. 


Answer  of  Data 

Data  Elewnt  Metric  Criteria  Factor 

El-"-"*  Score  Score  Score  Score 


FOURTH  MEASUREMENT  Date 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
aodules  that  are  coaponents  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


Ma-i 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

MAINTAINABILITY  MODULE 


INTRODUCTION  TO  MAINTAINABILITY 

The  Quality  Factor  Maintainability  represents  a  measure  of 
the  effort  required  to  locate  and  fix  an  error  in  an  operational 
program.  FIGURES  Ma  -  1  through  Ma  -  4  represent  the  hierarchy 
of  Metric,  Criteria  and  Quality  Factor  worksheets  associated 
with  Maintainability  in  Requirements  Analysis,  Preliminary  De¬ 
sign,  Detailed  Design  and  Implementation  phases  of  the  Software 
Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  BSD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Maintainability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


MMNtMNABlLm 


QUALITY  FACTOR 


FIGURE  Ma-3 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block-Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document . 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products. 


Ma-5 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/HO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0 . 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  /Jiswer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


Ma-6 


(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  *  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 
Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


ft. 
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HOW  TO  USE  SCORE  CHART  SETS 

When  you  apply  worksheets  to  system/subsystem/module, 
y.oa  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Li£e  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  $  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  ihe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 
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Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric.  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 
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Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set’s 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart’s  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 
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PRODUCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 
cation  |  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available,  see  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  -  PDR  minutes  and  action  item  responses 

NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 


DE¬ 

RfilRII 

TAILED 

FLOW 

CHART 

DATA 

FORMAT 

DESC. 

ALGO- 

RTHW 

DESC. 

NARY 

PRODUCT 

DESC. 

SYSTEM 

CPCI 
TEST 

TEST  |  PRO- 
PLANS 


SET 
TO  BE 
APPLIED 


ESD  PRODUCTS  FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 

•  Product  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

e  Product  B.  Preliminary  Product  Specifications, 

including  the  above 

e  Product  D.  CDR  minutes  ft  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 


DESIGN 

(UPDATE) 


PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 


Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
t  the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  coapiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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FINAL 

TO 


ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 

as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Teat  Procedures 

.  e  Product  2:  Segment  (i£  any)  and  system- level 

Test  Procedures 

B-D 

e  Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

•  Product  2:  Computer  Program  Coding  Changes 

Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

•  Product  3:  Modified  Pvoduct  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changes. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 
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DESIQJ  STRUCTURE  MEASURE 


Fora  Coda:  MaPEM.2 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SOURCECS): 

Q  SYSTEM 

□  SUBSYSTEM 

□  MXPLE 

UilA  . 

I.  DMA  COLLECntTM  raryfTfft:. 


1.0 

Hierarchical  Structure  (35) 

1.1  Is  a  hierarchical  chart  provided 
which  identifies  all  nodules  is 
the  system? 

Yes  •  1,  No  •  0 

Y  N 

□ 

2.0 

Module  Independence  (36) 

2.1  Is  the  nodule  independent  of  the 
source  of  the  input  or  the  destisa- 
tisn  of  the  output? 

Yes  ■  1,  No  •  0 

0D 

□ 

3.0 

Size  of  Data  Base  (70) 

3.1  Nuaber  of  unique  data  itans  in  date 
base. 

Score  "If  1 3.1 1 

□ 

□ 

II.  JCTRIC  WORJSECnON: 

«___  .....  .  Sun  of  Above  Scores 
Wth*  ■BoTonku  Eleaents 

□ 

III.  EVALUATION  WOgKSBCTIOW:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elenents  above? 
(1-10) _  (j>  If  you  are  unable  to  evaluate) 


□□ 


MAINTAINABILITY 


Complete  this  score  chan  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  B3  and  C  in  Analysis  Phase  are  available. 


Data  Answer  of  Data 
Element  Data  Element 

Number  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


FIRST  MEASUREMENT  Date 


Coaplete  this  score  chan  if.  a  second  application  of  this  set  of  worksheets  is 
appropriate  when  products  of  Q  and  C2  in  Design  Phase  are  available . 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Scon  Score 


SECOND  MEASUREMENT  Date 
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MAINTAINABILITY 


Complete  this  score  chart  if  a  third  application  of  this  set  of  worksheets  is  ap¬ 
propriate  when  products  of  Al,  A2,  and  B-Dl  in  Test  and  Integration  Ruse  are 
available. 


Data 

Element 

Number 


Answer  of  Data 

Data  Element 

Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


THIRD  MEASURB4ENT  Date 


Gonplete  this  score  chart  If  application  of  this  set  of  worksheets  is  appro¬ 
priate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration 
Phase  are  available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 

42 
144 

35 

36 
70 

FOURTH  MEASUREMENT  Date, 
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PROCEDURE  CONSISTENCY  MEASURE 


* 


% 


Form  Code:  MaDEM.l 


LIFE  'TOE  PHASE: 

DETAIL  DESIOJ 

SOURCE(S): 

□  SYSTEM 

□  SUBSYSTEM 

□  MXULE 

NM£: 

I.  DATA  QQILECTICN  WCRXSECTION: 

1.0  Standard  Design  Representation  (101) 

1.1  Does  the  design  representation 
comply  with  established  standards? 

Yes  -  1,  No  -  0 

2.0  Input /Output  Conventions  (102) 

2.1  Do  input/output  references  amply 
with  established  standards? 

Yes  •  1,  No  ■  0 

3.0  Calling  Sequence  Conventions  (103) 

3.1  Do  calling  sequences  amply  with 
established  standards? 

Yes  ■  1,  No  ■  0 

4.0  Error  Handling  Conventions  (104) 

4.1  Is  error  handling  done  according 
to  established  standards? 

Yes  ■  1,  No  ■  0 


II. 

METRIC  WRXSECnON: 

w_^_  _ Sum  of  Above  Scores 

Metric  Value  -  No.  oTPata' HScnts 

□ 

III. 

EVALUATION  NORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 

(1-10)  0?  If  you  are  unable  to  evaluate) 

IV. 

RtEPABED  BY:  APPROVED  BY: 

DATE: 

DAZE: 

□ 

□ 

□ 

□ 


OD 


DESIGN  STRUCTURE  MEASURE 


Fora  Code:  MaDEM.3 


I 


I 

I 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGJ 


3000(5): 


Q  SYSTEM 
SUBSYSTEM 
MODULE 


NNC: 


I.  DATA  COLLECTION  NCRKSECTICN: 


|  SCORE 


1.0  Module  Processing  not  Dependent  an  Prior 

Processing  (82) 

1.1  Is  the  module  independent  of  knowledge  , — 

of  prior  processing?  Y  N 

Yes  -  1,  No  -  0 

2.0  Entrance  and  Exit  of  the  Module  (83) 

2.1  Nimfcer  of  entrances  into  modules.  1  1 

2.2  Number  of  ex  its  from  module .  1  1 

Score-  1*  2.1  ♦  ^  *  jl*  |Z3]  *  lj 

3.0  Description  of  Input.  Output,  process  ins  and 

□ 

□ 

□ 

Limitations .  (138) 

3.1  Does  each  module  description  include  •— P-. 

input  .output .processing  and  limitations?  y  n 

Yes  ■  1,  No  »  0 

II.  METRIC  HORJSECnCN: 

.  Sum  of  Above  Scores 

Metric  Value  ■  no.  of  Data  Elements 

□ 

III.  EVALUATION  WDRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  Based  an  the  data  elements  above? 

(1-10) _  (0  if  you  are  unable  to  evaluate) 

PREPARED  BY; 


APPROVED  BY: 


□□ 


MODULAR  IMPLEMENTATION  MEASURE 


Form  Code: 


CYCLE  PHASE: 

DETAIL  DESIGN 


SCURCE(S): 


DATA  COLLECTION  WORKSECTION: 


SCORE 


1.0  Controlling  Pan-tars  (87) 

1.1  limber  of  calling  sequence  para¬ 
meters  that  are  control  variables. 

1.2  Nutoer  of  calling  sequence  para¬ 
meters. 


Scare  -  [IT]  ♦  1 1.2 1 


2.1  Is  input  passed  as  calling  sequence 
parameters? 


Yes  «  1,  No  •  0 


3.1  Is  output  data  passed  back  to 
calling  nodule? 

Yes  •  1,  No  -  0 

4.0  Controlling  Return  (90) 


4.1  Is  control  returned  to  calling 
nodule? 

Yes  -  1,  No  -  0 


Ma-27 


CONSISTENCY 


Fora  Code:  MaDDC. 1 


CYCLE  PHASE: 

DETAIL  DESIGN 


METRIC  SUMMARY  KSUCSECHON: 

1.  Procedure  Consistency  Measure 

2.  Data  Consistency  Measure 


III.  EVALUATION  MMgggCTjCN:  that  is  your  evaluation  of  the 
reviewed  products  based  an  the  nettles  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


SIMPLICITY 


Fozn  Code:  MaEDC.2 


CYCLE  PHASE: 

DETAIL  DESIST 


METRIC  SUWAKY 

1.  Design  Structure  Measure 

2.  Complexity  Measure 


II.  CRITERIA  WCRKSECTION: 


(1-10). 


Criteria  Value  ■ 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY:. 
DAZE: _ 
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MODULARITY 


Pot*  Cod*:  MaEDC.3 


CYCLE  ROSE: 

DETAIL  DESIGN 


SGORCECS): 

MaDDM.5 


1.  Modular  Inplementation  Measure 


_ _ St*  of  Above  Scores  I  I 

Cctfri*  Value  -  BoToTffiglcs - 1  I — » 


III.  EVALUATION  WOMSBCnCN:  Hhat  is  your  evaluation  of  the 
reviewed  products  bSied  on  the  aetrics  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


MAINTAINABILITY 


Foxn  Code:  MaEDF.l 


ILXFE  CYCLE  PHASE: 

DETAIL  DESIQJ 

rn  sYsiBi 


JOULE 


IMlCt 

rws.. 


S00Rf2(S): 

MaDDC.l 


MalSC. 2 


MaDDC.3 


I.  CRITERIA  SPMAKY  WBOBECnCN: 

1.  Consistency 

2.  Simplicity 

3.  Modularity 


□ 

III.  EVALUATION  WOWSBCTICN:  Hhat  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (|  If  you  are  unable  to  evaluate) 

HBPABg)  If:  APPROVED  BY: 

W3B: _  OKS: _ 
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□□□ 


H  Is  Is  Is  I5  N  Is  Is  Is  ¥  Is  Is  N  N  l§  ||g  j| 


MAIKIAJNABILITy 


*  this  score  chart  if  application  of  this  set  of  worksheets  is  appro- 
when  products  of  Al,  A2 ,  A3,  A4 ,  B  and  D  in  Design  Phase  axe  available 

Answer  of  Data 

Data  Eleaent  Metric  Criteria  Factor 


Score  Score  Score  Score 
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MADflAINABILm 


Complete  this  score  chart  if  a  second  application  of  this  set  of  Worksheets 
appropriate  for  update  when  products  of  C  in  Coding  and  Checkout  Phase  is 
available. 

Data  Answer  of  Data 

Element  Data  filemt  Metric  Criteria  Factor 

Kmfcer  Element  Score  Score  Score  Score 


MAINTAINABILITY 


Complete  this  score  chart  if  a  third  application  of  this  set  of  Worksheets  is 
appropriate  for  update  when  products  of  B-D3  and  3-D4  in  Test  and  Integration 
Phase  are  available. 

Answer  of  Data 

Data  Element 

Element  Score 


Data 

Element 

Nmber 


Metric 

Score 


Criteria 

Score 


Factor 

Score 


□□ 


CODING  SIMPLICITY  MEASURE 


Fota  code:  M&B***3 


CYCLE  PHASE: 

IMPLEMENTATION 


|SOURQ(S) 


Source  Code 


DATA  CDLLECTICN  WDRSECTION: 


native  Boolean  or 
pressicms  flOB 


Boolean 


1.1  Number  of  negative  or  complicated 
compound  Boolean  expressions. 

1.2  Number  of  lines  excluding  raiments. 
Score  «<l-(  1  l-ll  4*  QT7]) 

2.0  Statement  Labels  (108) 

2.1  Number  of  statement  labels? 

(Do  not  count  format  statements) 


2.7  Number  of  lines  excluding  a 
(Executable  statements) 

Score  »l-(  E2  +■  EH1 

3.0  Nesting  Level  (109) 

3.1  Maximum  nesting  level? 


Score  •□*113 


4.0  Conditional  Branches  (110) 

4.1  Number  of  conditional  branches? 


4.2  Nueber  of  executable  stai 


Score  •  1-  mi  flTil) 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY:. 
DATE: _ 
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5. 1  Number  of  unconditional  branches. 


S.2  Niaber  of  executable  statements . 

Score  ■  1-d  5.1  j  *  |s.2  1) 

6.0  In  and  Out  of  Loops  (112) 

6.1  Number  of  one  entrance/ 
one  exit  loops. 

6.2  Total  Niaber  of  loops. 

Score  -  6.1  t  1 6.2  | 

7.0  Loop  Index  (113) 

7.1  Niaber  of  loop  indices  that  are 
nodified. 


7.2  Total  number  of  loops. 

Score  "H  I  7.1  I  ♦  I  7.2  h 


□ 

□ 


□  □  □□ 


□□ 


DATA,  COLLECTION  WOWSECTTCW  fMiHwwrff : 

8.0  Module  Flow  Top  to  Bottom  CHS) 

8.1  Is  £low  top  to  bottom? 

There  should  not  be  any  backward 
branching  GOTOs.) 

Yes  •  1,  No  »  0 

9.0  Local  Variables  (128) 

9.1  Number  of  local  variables. 

9.2  Number  of  variables  (local  and  global) , 

Score  •  lV.il  +  1 9.2 1 


II.  FCIRIC  jCRKSECnON: 


Metric  Value 


Sub  of  Above  Scores  I  I 


III.  EVALUATION  NDRKSECTIOW:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


□o 


MODULAR  IMPLBffifrtAnCN 


Form  Cad*: 


002  PHASE: 

WPLBBfEAnGN 


i:v. 


1.0  Contro 


Piyeteri  (S7j 


1.1  Huber  -of  calling  sequence  peri¬ 
meters  that  are  control  variables. 


1.2  Number  of  calling  sequence 
meters. 

Score  ■  O-  flTTl 


2.1  Is  input  passed  as  calling 
parameters? 

Yes  -1,  Kb  ■  0 


3.1  Is  output  data  passed  back  to 
calling  module? 

Yes.  ■  1,  No  •  0 

4.0  Controlling  Return  (90) 


Metric  Value  • 


Sue  of  Above  Scores 
No.  of  Data  Element: 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(9  If  you  are  unable  to  evaluate) 


i  ?  5 1  -.*?  vJtMrill 


PREPARED  BY: 


APPROVED  BY: 
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□□ 


EFFECTIVENESS  OF 


Foot  Code: 


MaBM.6 


CYCLE  PHASE: 

iMPiaercAiiGN 


DATA  COLLECTION  VCRXSECTION 


Prologue  Conents  (118) 


1.1  Are  there  prologue  comments  contain¬ 
ing  information  about  the  function, 
author,  version  number,  data  inputs, 
outputs,  assumptions  and  limitations? 

Yes  ■  1,  No  ■  0 

Control  and  Destinations  Comment  (119) 


2.1  How  many  decision  points  and  trans¬ 
fers  of  control  are  not  commented? 

2.2  Total  number  of  decision  points. 

Score  ■!-  (f  2 . 1 1  •  |2 . 2  | ) 

Machine  Dependent  Code  Cce—at  (120) 


3.1  Is  all  machine  language  code 
commented? 

Yes  ■  1,  No  ■  0 

4.0  Non-standard  HOL  Statements  Comment  (121) 

4.1  Are  non-standard  HOL  statements 
commented? 


Yes  •  1,  No  •  0 


PREPARED  SY: 


APPROVED  BY: 


EFFECTIVENESS  OF  CXMCNTS  MEASURE  (Continued)  ■ 

Form  Code:  MaIVM'6 


CYO£  PHASE: 

IMPLBffiNEATICN 


DATA  CDLLECnCN 


Declared  Variables  Cemented  (122} 

5.1  How  many  declared  variables  are  not 
described  by.  Cannes ti? 

5.2  Nunber  of  variables. 


Score  -l-(  HD  *  H3> 


Lcfa  Do  Not 


6.1  Do  the  caenents  do  more  than  repeat 
the  operation? 

Yes  ■  1,  No  »  0 

7.0  Identification  of  Ccements  (137) 

7.1  Are  caenents  set  off  from  the  code 
In  a  uniform  manner? 

Yes  ■  1,  No  ■  0 


MsTRIC  jPMBBCnCN: 


Metric  Value  • 


Sum  of  Above  Scores 


III.  EVALUATION  WCRKSECTION:  Nhat  is  your  evaluation  of  the 
reviewed  products  Eased  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


□□ 


descriptiveness  of  impl^entatign  language 


Fora  Coda: _ Ma 


CYCLE  PHASE:  SOURCE (S) : 

IMPL94ENIATIGN 


DATA  COLLECTION  VCRXSECnON: 


1.0  High  Order  Language  (107) 


1.1  Is  high  order  language  used? 
If  Yes  •  1,  No  -  0 


2.0  Variable  Names  (123) 


2.1  Are  variable  name  (mnamonices) 

descriptive  of  the  physical  or  ■— j— . 

functional  property  they  represent?  |y|N| 

If  Yes  ■  1,  No  *  0 


3.0  Blocked  and  Indented  Source  Code  (125) 

3.1  Is  the  code  logically 
blocked  and  indented? 

If  Yes  ■  1,  No  ■  0 


PREPARE)  BY: 
DATE: _ 


APPROVED  BY:, 
DAZE: _ 


DO 


lue  am  phase: 

IMPLBENTAXICN 

SCURCE(S) : 

nt 

UUM. 

I.  .  DATA  OQLLSCnCN  HORKSECTICN  (continued) : 

4.0  One  Statement  Par  Lina  (126) 

4.1  Haber  of  lines  with  more  that  1  1 

ana  statement.  _ 

4.2  Haber  of  continuous  lines.  1  1 

4.3  The  total  number  of  lines  in  a  ; - 1 

-£..x-(fcfe.O)  D 

1  □ 

tt.  ACTRIC  ernrsRrrrnw- 

..  _  _  Sum  of  Above  Scores 

mtgic  Valu»  *  NoTof  Bata  BeBBEs 

□ 

III.  EVALUATION  NORXSBCTZCM:  Mat  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  el— anti  above? 

(1-10)  (0  If  you  are  —able  to  evaluate) 

□□□ 


□o 


SIMPLICITY 


RISE  cm  PHASE: 

IMPLEMENTATION 


Us55m" 


NAM5:_ 


I. 


jcnuc  awg  WOMESECTION: 


Foia  Cod>:-_MaDjC1li 

SOURCE  (S) : 

MaIKM.1  MaIMM.2 


MaBM.3 


SCORE 


1.  Design  Structure  Measure 

2.  Complexity  Measure 

3. ’  Coding  Simplicity  Measure 


n. 


III. 


QtTIERIA  NCRKSECTION: 


Criteria  Value  - 


Sue  of  Above  Scores 

no.  of  Metrics 


EVALUATION  WORKSBCTION:  Rhat  is  your  evaluation  of  the 
reviewed  products  based  on  the  netrics  above? 

(1-10)  (|  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY:. 
DATE: _ 


DATE 


□□□ 


I 


MODULARITY 

Form  Code:  MaIMC.2 

LIFE  CYCLE  PHASE: 

IMPLBENTATKU 

SOURCE  (S) : 

MaDM.4 

ill 

□□□ 

uiyc « 

I.  MEXRIC  SUMMARY  WRXSECTTQN: 

1.  Modular  Ispleaentatian  Measure 

. 

SCORE 

□ 

II.  GUTERIA  WCRKSECnON: 

. .  Su*  of  Above  Scores 

Criteria  Value  'feTof  IBtfgs - 

□ 

III.  EVALUATION  WURESBCTION:  Nhat  is  your  evaluation  of  the 
reviewed  products  based  on  the  astrics  above? 

(1-10)  (f  If  you  are  unable  to  evaluate) 

BMP  1Y:  AFROV8D  JY: 

MU:  _  QA2B: _ 


Ma-48 


1 


■w* 


SELF-DESCRIPTIVENESS 

Form  Code:  MaIMC.3 

LIFE  CYCLE  PHASE: 

IMPLBENTATICN 

SOORffiCS): 

MaBM.5  MaBW.6 

□  srsrm 

□  SUBSYSTBf 

□  MCCGLE 

MAMP* 

MaDW.7 

I.  METRIC  SUtiARY  WMSECTION: 

1.  Quantity  of  - consents  aeasure 


2.  Effectiveness  of  consents  measure 


III. 


3.  Descriptiveness  of  implementation 
language  aeasure 


CRITERIA  JCMggCnCN: 

Criteria  Value 


Sun  of  Above  Scores 
■ffc.  of  Metrics 


EVALUATION  RCRKSBCTIGN:  What  is  your  evaluation  of  the 
reviewed  products  bSsad  on  the  metrics  above? 

(1*101  (0  If  you  are  unable  to  evaluate) 


PREPARED  If: 


APPROVED  BY: 


□  □□ 


MAINTAINABILITY 

Form  Coda:  MaIMF.1 

LIFE  CYCLE  PHASE: 

IMPLEMENTATION 

SOURCS(S) : 

MaIMC.1  MaIMC.2 

□□□ 

NAME: 

MaIMC.3 

I.  CRITERIA  SUMMARY  WGRJCSECTICN: 

1.  Simplicity 

2 .  Modularity 

3.  Self-Descriptiveness 

□  □□  1 

II.  FACTOR  WORJSECTICN: 

„  _  ...  Sum  of  Above  Scores 

fetor  Vtlm  •  fe.  of  Criteria - 

□ 

III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  an  the  criteria  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 

PREPARED  BY:  APPROVED  BY:, 

DAIS: _  DATE: _ 


Ma-50 


MAINTAINABILITY 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro 
priate  when  products  of  A  and  B  in  Coding  and  Qieckout  Phase  arc  available . 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Scote  Score  Score 


15  N  M 


MAINTAINABILITY 


MAINTAINABILITY 


Sjl  a|  i  a|  aM  g|  at  s)  N  N  N  4 


MAINTAINABILITY 


Answer  of  Data 

Data  El  merit  Metric  Criteria  Factor 

Element  Score  Score  Score  Score 


M£-54 


Data 

Element 

Number 


Answer  of 
Data 
Element 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  ’’COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 
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INTRODUCTION  AND  INSTRUCTIONS 
FOR 

FLEXIBILITY  MODULE 

INTRODUCTION  TO  FLEXIBILITY 

The  Quality  Factor  Flexibility  represents  a  measure  of  the 
effort  required  to  modify  an  operational  program.  FIGURES  Fx  - 
through  Fx  -  4  represent  the  hierarchy  of  Metric,  Criteria 
and  Quality  Factor  worksheets  associated  with  Flexibility  in 
Requirements  Analysis,  Preliminary  Design,  Detailed  Design 
and  Implementation  phases  of  the  Software  Development  Life 
Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


Fx  - 1 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Flexibility  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 


Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block-Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block-On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Comp  .sr  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products. 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  (J. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 
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(5.2) 

(5.3) 

(5.4) 

(5. 5) 

(5.6) 

Step  6  - 
(6.1) 

(6.2) 

(6.3) 

(6.4) 

(6.5) 

(6.6) 


Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet, 

Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

FACTOR  WORKSHEETS 

Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


Fx-6 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


HOW  TO  USE  SCORE  CHART  SETS 


When  you  apply  worksheets  to  system/subsystem/module, 
you  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  $  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Cl. 3) 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located. the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com* 
pact  form. 


PDR 

MINUTES 

AND 

ACTION 

nBi 


PRODUCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  ot  Authenticated  System  Specifi¬ 
cation  I  Segment  Specification  (if  any) 

Product  B.l  -  Authenticated  Development  Specification  foT 
each  CPCI 

e  Product  B.2  -  Possible  higher- level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B. 3  •  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 
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FUNC¬ 

TIONAL 

FLOW 

CHARTS 


DATA  ALGO-  NARY 
FCW4AT  RITTW  PRODUCT 
DESC.  DESC.  DESC. 


SYSTB4 

CPCI 
TEST 

TEST  |  PRO- 
PLANS 


FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 
e  Product  A. 4:  Descriptions  of  algorithms  not 


Product  B. 


e  Product  D.  CDR  minutes  I  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 


previously  prescribed 

Preliminary  Product  Specifications, 
including  the  above 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


PRODUCTS 

CODE 

ALTERED  PRODUCT  SPECS 

S 

DcraercATicN 

DETAILED  DESIGN 

xcmacEis 

WORKSHEETS  (UPDATE) 

ESP  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  »  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor -supplied  information  should  be  searched. 
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PRODUCTS  FOR  TEST  AND 


INTEGRATION  PHASE 


Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Test  Procedures 

.  e  Product  2:  Segment  (if  any)  and  system-level 

Test  Procedures 

B-D 

e  Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

•  Product  2:  Computer  Program  Coding  Changes 

Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 
B-D 

e  product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changes. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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DATA  STORAGE  EXPANSION  MEASURE 


Form  Code: 


LIFE  CYCLE  PHASE: 


ISCURCE(S) : 


DETAIL  DESIOi 


SY5TB4  NAME 


DATA  OOLLECnON  WORKSECTION: 


SCORE 


1.1  Is  logical  processing  independent 
of  storage  specification? 


Yes  -  1,  No  •  0 
•Score  of  the  Systea  • 


No.  of  Modules  with  "YES" 


II.  METRIC  WORXSECTICN: 


Metric  Value 


Sun  of  Above  Scores 


III.  EVALUATION  WORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  eleoents  above? 
(1-10) _  (9  If  you  are  unable  to  evaluate) 


□□ 


MODULAR  IMPLEMENTATION  MEASURE 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 

□□□ 

NAME: 

Form  Code;  FxDEM.3 


SCJURCE(S): 


I.  DATA  COLLECTION  VOUSEOTON: 

1.0  Controlling  Parameters  (87) 

1.1  Number  of  calling  sequence  para¬ 
meters  that  are  control  parameters. 

1.2  Number  of  calling  sequence  para¬ 
meters. 

Score  ■  ll.l  1  +  1.2 
2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  -  1,  No  -  0 

3.0  Controlling  Outixxt  (89) 

3.1  Is  output  data  passed  back  to 
calling  module? 

Yes  ■  1,  No  ■  0 

4.0  Controlling  Return  (90) 

4.1  Is  control  returned  to  calling 
module? 

Yes  ■  1,  No  ■  0 


□ 

□ 


YlN 


YIN 


SCORE 


□ 


□ 


□ 


□ 


PREPARED  BY: 
DATE; _ 


APPROVED  BY: 

m 

DATE: _ 
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MODULAR  IMPLiMENTAXION  MEASURE  (Continued) 

Form  Code:  FxDEM.3 


JFE  CYCLE  PHASE: 

DETAIL  DESIGN 
UJ  SYSTBi  "  NAME:. 

□  SUBSYSTEM  - 

□  MODULE  - 

I.  DATA  COLLECTION 


SQURQ(S) : 


MBagSagil 


5.0  Share  Temporary  Storage  (91) 

5.1  Is  temporary  storage  independent  of 
other  modules? 

Yes  ■  1,  No  ■  0 


II.  METRIC  WORKSECHON: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


.i\,x'a.TTr^ 


IV.  INSPECTOR'S  OCjfCKrS: 


PREPARED  BY: 
SATE: _ 


APPROVED  BY: 
DATE: _ 


Px-18 


.•USiVv.. 


□□L 


GENERALITY  CHECKLIST 


Form  Code:  FxDDM.4 


DATA  OOLLECnON  WORKSECTION 


1.1  Does  the  Module  not  mix  input, 
output,  end  prodSSBing  functions 
in  the  sane  nodule? 


Yes  -  1,  No  -  0 

2.0  Machine  Dependent  Functions  (931 

2.1  Nuber  of  machine  dependent  functions 

performed.  _ _ _ 

Score  •  1  +  1 2.1 1 

3.0  Unlimited  Data  Volume  (94) 

3.1  Is  processing  data  volume  imHnritwi? 
Yes  •  1,  No  •  0 

4.0  Unlimited  Data  Value  (95) 

4.1  In  the  Design  Phase,  is  processing 
data  Value  imlinrirmd? 

Yes  •  1,  No  -  0 


II.  METRIC  WORKSECTION: 


Metric  Value 


Sum  of  Above  Scores 


PREPARED  BY: 


APPROVED  BY: 
DATE: _ 


Fx-19 


f 


EXPANDABILITY 


JPE  CYCLE  PHASE: 
_ DETAIL  DESICfl 

□  SYSTBi  NMC:. 

□  SQBSYSXBt  - 

□  MODULE  - 


SOORCE(S) : 
FxDEM.l 


Form  Code:  FxDDC.l 


FxDEM.2 


1.  Date  Storage  Expansion  Measure 

2.  Extensibility  Measure 


II.  CRITERIA  KCRXSECnON: 


Sum  of  Above  Scores 
Criteria  Value  -No.  6F  Wtrics - 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  metrics  above? 

(0  If  you  are  unable  to  evaluate) 


■^T^rarrrr^ 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY:. 
DATE:  _ 


Fx-20 


MODULARITY 


Form  Code: 


LIFE  CYCLE  PHASE:  SOURCE (S) : 

DETAIL  DESIGN  FxECM.3 

H  SYSTEM  Fh5£:  . 

□  SUBSYSTBi  -  - 

□  MODULE  - 

I.  METRIC  SflMARY  WORKSECTION: 

1.  Modular  Iaqplementation  Measure 


II.  CRITERIA  MORJSECTION: 


«..  .  ...  Sun  of  Above  Scores  r“”| 

Criteria  Value  -  fo.  0£  Metrics - 1  I - 1 


III.  EVALUATION  WORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  Eased  on  the  metrics  above? 

(1-10) _  (jj  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY:. 
DAIS: _ 


Fx-21 


□□I 


GENERALITY 


Fora  Cock:  FxDDC.3 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 

SCORCECS): 

FxDEM.4 

□□□ 

HUMt* 

I.  METRIC  SMIARY  WDRXSECTICN: 

1.  Generality  Checklist 

• 

SCORE 

□ 

II.  CRITERIA  WCRKSECTICN: 

_  ..  .  ...  Sun  of  Above  Scores 

Criteria  Value  "fo,  oTSfetrics - 

□ 

III.  EVALUATION  WORKSECTICN:  What  is  vour  evaluation  of  the 
reviewed  products  based  on  the  aetrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 

PREPARED  BY: 
001: _ 


APPROVED  BY: 

« 

DttB: _ 


FLEXIBILITY 


1.  Expandability 

2.  Modularity 


III 


□□□ 


FLEXIBILITY 


Complete  this  score  dart  if  a  second  (or  iterative)  application  of  this  set 
of  tnrksheets  is  appropriate  for  qpdate  when  the  products  of  C  in  Coding  and 
Checkout  phase  becomes  available. 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Nmiber  Element  _ Score  Score _  Score _  _ Score 


MODULAR  IMPLEMENTATION  MEASURE 


?3ZB  Code:  FxINM.l 


CYCLE  PHASE: 


(SOURCE  (S): 


IMPLEMENTATION 


DATA  03LLECITCN  M3RKSECTICN: 
1*0  Controlling  Parameters  (873 


1.1  Nunfcer  of  ral1 1  sequence  pen* 
meters  that  are  control  variables. 

1.2  Number  of  1  sequence  pan* 

meters.  _  _ _ 

Score  • mi  ♦  mu 
2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  •  1,  No  ■  0 

3.0  Controlling  Output  (89) 

3.1  Is  output  data  passed  back  to 
calling  nodule? 

Yes  •  1,  No  -  0 

4.0  Controlling  Return  (90) 

4.1  Is  control  returned  to  calling 
nodule? 

Yes  -  1,  No  -  0 


II.  METRIC  NCTOCSBCnCN: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


d-io). 


What  is  your  evaluation  of  the 
an  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 
DATE: _ 


Fx-27 


GENERALITY  CHECKLIST 


Font  Cods:  FxIMM.2 


CYCLE  ffifcSS: 

IMPLEMENTATION 


□  SOBSYSTB4 

□  MXCLE 


DATA  gJLLECnON  TOUSECTION: 


1.1  Does  the  nodule  not  nix  input, 
output,  end  processing  functions 
in  the  same  nodule? 

Yes  -  1,  No  •  0 

2.0  Machine  Dependent  Functions  (93) 

2.1  Nunber  of  machine  dependent  functions 
performed.  . 

Score  ■  1  ♦  1 2.1 1 

3.0  Unlimited  Data  Volune  (94) 

3.1  Can  the  amounts  of  data  that  can 
be  processed  be  uni  ini  ted? 

Yes  ■  1,  No  •  0 

4.0  Unlimited  Data  Value  (95) 

4.1  In  the  Isplementation  Phase,  is  the 
value  of  data  that  can  be  processed 
unlimited? 

Yes  •  1,  No  -  0 


II.  ICIRIC  WCMBECnOW: 


Metric  Value 


Sum  of  Above  Scores  j  j 

■  V17~  "IX  I— J 


III.  EVALUATION  WCRKSECnON:  What  is  your  evaluation  of  the 
reviewed  products  ba55d  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


>  Me. 


EXTENSIBILITY  MEASURE 

Form  Code:  FxD-M.3 

LIFE  CYCLE  PHASE: 

IMPLEMEN] 

CATION 

SOURCE (S) : 

Q  STSTBi 

□  SUBSYSTEM 

□  MODULE 

mm: 

I.  DATA  COLLECTION  WORXSECTION: 

1.0  Accuracy.  Convergence  and  Timing  Attributes  (84) 

1.1  Are  accuracy,  convergence,  or  timing  pi— 

attributes  parametric?  Y  N 

Yes  ■  1,  No  •  0 

SCORE 

□ 

II.  METRIC  WJRKSECTION: 

- -  ....  .  .  Sum  of  Above  Scores 

Metric  Value  -  fo.  0f  feta  TlmSnts 

□ 

III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  an  the  data  elements  above? 

(1-10)  (?  If  you  are  unable  to  evaluate) 

PREPARED  BY: 


APPROVED  BY: 


ODL 


EFFECTIVENESS  OF  COMMENTS  MEASURE 

Form  Code: 


FxBM.S 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


SOURG(S): 


□  SYSTEM 

□  SUBSYSTEM 

□  MCCUIE 


NAME: 


DATA  COLLECTION  VCRX5ECTI0N: 

1*0  Prologue  Comnents  (118) 

1.1  Are  there  prologue  aments  contain  - 
ning  information  about  the  function, 
author,  version  amber,  date  inputs, 
outputs,  assumptions  and  limitations? 

Yes  -  1,  No  -  0 

2.0  Control  and  Destinations  Comment  (119) 

2.1  How  many  decisions  points  and  transfers 
of  control  axe  not  commented? 

2.2  Total  number  of  decision  points. 

Score  »1- d £1 1  +  2.2  ) 

3.0  Machine  Dependent  Code  Cement  (120) 

3.1  Is  all  machine  language  code 
commented? 

Yes  -  1,  No  -  0 

4.0  Non-standard  HJL  Statements  Comnent  (121) 

4.1  Are  non-standard  HQL  Statements 
Goonented? 

Yes  -  1,  No  -  0 


I 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 


Fx-31 


EFFECTIVENESS  OF  COMMENTS  MEASURE  (Continued' 


Fom  Code;  FxB>M.  5 


CYCLE  PHASE: 

IMPLEMENTATION 


|S0URCE(S): 


DATA  COLLECTION 


S.o  Declared  Variables  Connented  (122) 

5.1  How  many  declared  variables  are  not 
described  by  cements? 

5.2  Nrnber  of  variables. 

Score-1-  dm  +  imp 

6.0  Conments  Which  Do  Not  Only  Repeat  the 
Operation  (124] 

6.1  Do  the  cements  do  sere  than  repeat 
the  operation? 

Yes  -  1,  No  -  0 

7.0  Identification  of  Comments  (137) 

7.1  Are  cements  set  off  fron  the  code 
in  a  uniform  manner? 

Yes  -  1,  No  -  0 


II.  METRIC  WORJ5ECTION: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


DATE: 


DATE: 


Fx-32 


DESCRIPTIVENESS  OF  IMPLEMENTATION  LANGUAGE 


Form  Code:  FxDW.6 


CYCLE  PHASE: 

IMPLEMENTATION 


NAME: _ 


DATA  COLLECTION 


1.1  Is  high  order  language  used? 

Yes  •  1,  No  •  0 

2.0  Variable  Nanas  (123) 

2.1  Are  variable  naae  (menonices) 
descriptive  of  the  physical  or 
fimr-Hnnfti  property  they  represent? 

Yes  *  1,  No  •  0 

3.0  Blocked  and  Indented  Source  Code  (125) 

3.1  Is  the  code  logically  blocked  and 
indented? 

Yes  ■  1,  No  ■  0 


□□ 


□  □□ 


MODULARITY 


Pom  Code:  FxIMC.l 


CYCLE  PHASE: 

IMPLEMENTATION 


SOURCE (S] : 
FxOM.l 


I.  METRIC  SUMMARY 


1.  Modular  Implementation  Measure 


II.  CRITERIA  WORKSECTION: 


Criteria  Value 


Sum  of  Above  Scores 


No.  of  Metrics 


III.  EVALUATION  WORKSECHON:  tfhat  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 
DAIS: _ 


APPROVED  BY: 

m 

DAZE: _ 


Fx-35 


GENERALITY 


LIFE  CYCLE  RUSE: 

IMPLEM1 

3NTATICN 

ODD 

UUflt 

For*  Cods;  FxIMC.2 


FxBM.2 


I.  METRIC  SOM, KY  HOMBECTION: 

1.  Generality  Checklist 


SCORE 


□ 


•II. 

CRITERIA  BOaSBCnCN:  ”"l 

Criteria  Vain.  .  8*  tf)0v?J$C0r#! 

□ 

criteria  value  -  }£  of  Metrids 

III.  EVALUATION  WCRKSECnOM;  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  aetrics  above? 

(1-101  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


□□ 


□□ 


SELF-DESCRIPTIVENESS 


Font  Code:  FxIMC.4 


KEIRIC  SUMMARY 

1.  Quantity  of  Ccmmwts  Measure 

2.  Effectiveness  of  Ownts  Measure 


3.  Descriptivess  of  Implementation  Language  Measure 


□ 

□ 

□ 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION  MURKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (|  If  you  are  unable  to  evaluate) 


I 


i 


FLEXIBILITY 

Fozn  Code:  FxIMF.l 

LIFE  CYCLE  PHASE: 

IMPLEMEOTATICN 

SOURCE  (S) : 

FxIMC.l  FxIMC.2 

Q  SYSTB4 

□  saBsarat 

□  >am 

4|AWP  - 

ExIMC.4  FxIMC.3 

I.  CRITERIA  SttMAKY  WCRKSECTION: 
1.  Nodularity 


2.  Generality 

3.  Expandability 

4 .  Self -Descriptiveness 


II. 

FACTOR  DIORXSECnON: 

- -  -  Sum  of  Above  Scores 

Factor  Value  -  fo.  0f  Criteria - 

□ 

III. 

EVALUATION  WORKSECTIGN:  What  is  vour  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 

HEPARED  BY:  APPROVED  BY: 

DAS:. 

DAS: 

Fx-39 

□□□□ 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


SSSffSf" 


Te-i 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

TESTABILITY  MODULE 


INTRODUCTION  TO  TESTABILITY 

The  Quality  Factor  Testability  represents  a  measure  of  the 
effort  required  to  test  a  program  to  insure  it  performs  its 
intended  function.  FIGURES  Te  -  1  through  Te  -  4  represent  the 
hierarchy  of  Metric,  Criteria  and  Quality  Factor  worksheets 
associated  with  Testability  in  Requirements  Analysis,  Prelimin¬ 
ary  Design,  Detailed  Design  and  Implementation  phases  of  the 
Software  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets,  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Testability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


I 


FIGURE  Te-3 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block -Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products. 


Te-5 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(S.l)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


Te-6 


(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric.  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet . 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


Te-7 


STEPS  FOR  UPDATE  APPLICATIONS 
Step  1 


Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


HOW  TO  USE  SCORE  CHART  SETS 


When  you  apply  worksheets  to  system/subsystem/module, 
y.oa  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  ChaTt  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  -locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  £he  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  •  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
fum  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  ScoTe". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


Te-10 


Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


TRADE¬ 

OFF 

STUDY 


INITIAL  OR 


PROHJCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  4  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPC I 

e  Product  B.2  -  Possible  higher- level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  -  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 


FUNC¬ 
TIONAL 
FLOW  I  FLOW 
CHARTS  I  CHART 


DATA  ALOO-  I  NARY 
FORMAT  I  RT1H1  |  PRODUCT 
DESC. 


SYSTBf 

CPCI 
TEST 
TEST  I  PRO- 


CTS  FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 
e  Product  A. 4:  Descriptions  of  algorithms  not 


Product  B. 


Product  D. 


previously  prescribed 

Preliminary  Product  Specifications , 
including  the  above 

CDR  minutes  I  action  item  responses 


Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note, 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor- supplied  information  should  be  searched. 


ALTERS)  PRCDUCT 


DETAILED  DESIGN 
WORKSHEETS  (UPDATE) 


PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 


Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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PRODUCTS 

FINAL 

CPCI 

TEST 

PROS- 

TEST 

REPORTS 

CODING 

CHANGES 

MODIFIED 

PRODUCT 

SPECS 

m 

uses 

mtoerbbpt 

DffLBCN- 

SET  TO  BE 

TATZGN 

APPLIED 

WCRKSSSTS 

WORKSHEETS 

(UPDATE) 

ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 


Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Tost  Procedures 

e  Product  2:  Segment  (if  any)  and  system- level 
Test  Procedures 

B-D 

e  “Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  “Product  2:  Computer  Progrpm  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changes. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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DESIGN  STRUCTURE  MEASURE 


Fora  Code:  TePEM.l 


SOURCE(S); 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGI 


SYSTEM  NAME: _ 

□ 


I.  DATA  COLLECTION  WORKSECHON: 


1.0  Hierarchical  Structure  (35) 

1.1  Is  a  hierarchical  chart  provided 
which  identifies  all  nodules  in 
the  system? 

Yes  -  1,  No  -  0 

2.0  Module  Independence  (36) 

Z.  1  Is  the  nodule  independent  of  the 
source  of  the  input  or  the  desti» 
nation  of  the  output? 

Yes  -  1,  No  -  0 

3.0  Site  of  Data  Base  (70) 

3.1  Ni ater  of  unique  data  items  in  data 

base.  . 

Score  •  1  +  13.1 


II.  METRIC  WORKSECHON: 


III.  EVALUATION  WDIUCSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  eleneats  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


Metric  Value  ■ 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY: 
DATE: _ 
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MODULE  TESTING  MEASURE 


Form  Code:  TePEM.2 


CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SOURCE (S) : 


nATA  miJECnON  WORKSECTION: 


1.0  Path  Coverage  (64) 

1.1  Number  of  paths  to  be  tested. 

1.2  Total  mriber  of  paths. 

Score  ■  I  111  1.2 

2.0  input  Parameters  Boundary  Tested  (65) 

2.1  Number  of  input  parameters  to  be  tested 

2.2  Total  amber  of  input  parameters. 

Score  ■  1 2.1 1  +  12.2 


B 

•B 


METRIC  WORKSECTION: 


Metric  Value 


Sum  of  Above  Scores 
No.  of  Data  Elements 


III.  EVALUATION  WORKSECTION:  Whet  is  your  evaluation  of  the 
reviewed  products  based  on -the  data  elements  above? 
(1-10) _ Of  If  you  are  unable  to  evaluate) 


□□ 


SYSTEM  TESTING  MEASURE 


< 


Foot  Code:  TePEM.4 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIOJ 

SOURCE(S): 

Q  SYSTB4 

0  SUBSYSTEM 

O  M3XJLE 

me: 

I.  DATA  COLLECTION  W3RKSECTI0N: 

1.0  Module  Coverage  (68) 

1.1  Number  of  modules  to  be  exercised. 

1.2  Number  of  modules. 

Score  •  ll.ll  +  1 1.2 1 


2.0  Identification  of  Test  Inputs  and  fotputs  in 

femaTy  farm  T691 


2.1  Are  tost  inputs  nd  outputs  provided 
in  Jinmi  y  form? 

Yes  *  1,  No  -  0 


METRIC  WQRKSECTICN: 

Metric  Value 


Sub  of  Above  Scores 
No.  of  Data  Elements 


III.  EVALUATION  WCRKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10)  (9  If  you  are  unable  to  evaluate) 


IV.  INSPECTOR'S  oagens: 


PREPARED  BY: 


APPROVED  BY: 


□□ 


INSTRUMENTATION 

Form  Code: _ 

TePDC.2 

SOURS  (S): 

TePEM.2 

TePEM.3 

METRIC  SUMMARY 


1.  Module  Testing  Measure 

2.  Integration  Testing  Measure 


II.  CRITERIA  WDRK5BCTI0N: 


Criteria  Value* 


Sum  of  Above  Score 


III.  EVALUATION  WBUSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  bSed  on  the  metrics  above? 

(1-10) _  (0  If  you  are  unable  to  evaluate) 


Te-21 


□□□ 


SYSTEM  TESTING  SUPPORT 

Pom  Code:  TePDC.3 

LIPS  CYCLE  PHASE: 

PRELIMINARY  DESI04 

SOURCE (S): 

TePEM.4 

□  SYSTB4 

O  SQBSYSQM 

□  MODULE 

NHG: 

veaiinv  MiiirPL1/  T 1 1  mr. 

SCORE 

UMARZ  mjKKScUi  1UN I 

1.  System  Testing  Measure 

• 

□ 

II.  FACTOR  NDRJSECriON: 

_ _  .  Sum  of  Above  Scores 

Factor  Value  ■  Nb/  of  Siteria - 

□ 

III.  EVALUATION  KOUBBCriCN:  Nhat  is  your  evaluation  of  the 

reviewed  produetiTSSed  on  tne  criteria  above? 

(1-10)  (F  If  you  are  unable  to  evaluate) 

PREPARED  BY: _ 

APPROVED  BY: 

TESTABILITY 


Form  Code: 


TePDF.l 


CYCLE  PHASE: 


PRELIMINARY  DESIGN 


( S): 

TePDC.l 


TePDC.2 


CRITERIA  SIM4AKY  NCRKSECHCN 


1.  Simplicity 

2.  Instrumentation 

3.  System  Testing  Support 


FACTOR  jCWBECnCN: 


Factor  Value  ■ 


Sue  of  Above  Scores 


III.  EVALUATION  WORCBCnON:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


APPROVED  BY: 


□□□ 


TESTABILITY 


Complete  score  chart  if  a  second  application  of  this  set  of  worksheets 
is  appropriate  for  update  when  products  of  Q  and  C2  in  Design  ffcase  are  availa¬ 
ble. 


Data  Answer  of  Data 
Element  Data  EL— eat 

N—ber  El— ent  Score 


Metric  Criteria  Factor 

Score  Score  Score 


35 

36 
70 

64 

65 


66 


TESTABILITY 


Conplet*  this  socre  chart  if  a  Fourth  application  of  this  sat  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration 
phase  are  available! 


Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 
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□□ 


□□ 


MODULAR  IMPLEMENTATION  MEASURE 


Font  Code:  TeDEM.3 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 


□  SYSTEM 

□  SUBSYSTEM 

□  M33ULE 


I.  DATA  COLLECTION  WJRESECITON: 


1.0  Control 


Parameters  (87) 


1.1  Number  of  calling  sequence  para¬ 
meters  that  are  control  variables. 

1.2  Number  of  calling  sequence  para¬ 
meters. 


’  Score  *  £3  -  mi 


2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  ■  1,  No  ■  0 


3.1  Is  output  data  passed  back  to 
calling  module? 

Yes  •  1,  No  •  0 
4.0  Controlling  Return  (90) 


4.1  Is  control  returned  to  calling 
module? 


Yes  •  1,  No  -  0 


PREPARED  BY: 


APPROVED  BY: 

ef 

DATE: _ 


Te-30 


MODULAR  IMPLEMENTATION  MEASURE 


Fora  Cod*:  TeDDM.3 


5.1  Is  temporary  storage  independent  of 
other  modules? 

Yes  -  1,  No  ■  0 


II.  METRIC  NORKSECTION: 


Metric  Value 


_  Sub  of  Above  Scores  I  i 


III.  EVALUATION  WOttSBCTICN:  *bat  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  eleaents  above? 
(1-10)  (0  If  you  are  unable  to  evaluate) 


□□I 


SIMPLICITY 


Form  Code: 


TeQDC.l 


lUFE  CYCLE  PHASE: 

DETAIL  DESIQ4 


nr  SYSTB4 

SUBSYSISM 
MODULE 


UAVB  . 

1WE  • 


SOURCE  (S) : 

TeDDM.2 


I. 


METRIC  SUMMARY  VDRKSECTIQN: 


TeDEM.l 


SOBBS 


II. 


III. 


1.  Complexity  Measure 

2.  Design  Structure  Measure 


□ 

□ 


CRITERIA  WORXSECriON: 


Criteria  Value 


Sum  of  Above  Score; 
Tto.  of  Metrics 


EVALUATION  W0R1SBCTT0N:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY 


APPROVED  BY: 


MODULARITY 


Form  Code:  TeDDC.2 


CYCLE  PHASE: 

DETAIL  DESIGN 


SOURCE  (S) : 

TeDEM.3 


METRIC  SUMMARY 


1.  Modular  Implementation  Measure 


II.  CRITERIA  WURKSECTION: 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
on  the  ae tries  above? 

(0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 
DATE: _ 


Te-33 


TESTABILITY 


Fora  Code:  TeDDF .  1 


CYCLE  PHASE: 

DETAIL  DESIGN 


SOUO(S): 

TeBDC.l 


TeEDC.2 


CRITERIA  SUGARY  WORKSECTION 


1.  Simplicity 

2.  Modularity 


II.  FACTOR  NDRKSECnCN: 


Factor  Value  ■ 


Sun  of  Above  Scores 


□ 


III.  EVALUATION  M3RKSECTICN:  What  is  your  evaluation  of  tbs 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


3 


DESIGN  STRUCTURE  MEASURE 


I 


(LIFE  CYCLE  PHASE: 

IMPLEMENTATION 
fD  SYSTH4 

□  SUBSYSTEM 

□  MODULE 


Fora  Code:  TeDM.l 


IWB*. 


!S0URQ(S): 


I. 


DATA  COLLECTION  WCRKSECTION: 

1.0  Entrance  end  Beit  of  the  Module  (83) 

1.1  Number  of  entrances  into  nodules. 

1.2  Nunfcer  of  exits  from  nodule. 

Scare  • 


-1-  .  -L 


ED  ED 


II.  METRIC  WORXSECriGN: 


Metric  Value 


Sun  of  Above  Scores 
Sol  of  Data  Elasents 


SCORE 


□ 


□ 


III.  EVALUATION  WCRKSBCTION:  What  is  your  evaluation  of  the 
reviMoa  products  oaieid  on  the  data  elewots  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


IV.  INSPECTOR* S  COHgNTS: 


PREPARED  BY:. 
DA31: _ 


APPROVED  BY:. 
DATS: _ 


Te-37 


~  DO 


OCMPLEXITY  MEASURE 


Foza  Code:  TeDW.2 


SY5TB4 


'lifts 


1.0  Pete  end 


si  Flow  Coonlexity  (81) 


1.1  Whet  is  the  sia  of  the  number  of 
decision  points,  suhdecisien  points, 
breaches  vncoiidi- 
tional  breaches? 


Score  •  1  ♦ 


[Q  ♦  i] 


_ Sun  of  Above  Scores  I  I 

Metric  Value  ■  n. -»  mux  1 - I 


in.  EVALUATION  WCRJSBCHCN:  feat  is  your  evaluation  of  the 
reviewed  products  Sued  on  the  data  elenents  above? 
(1*10)  (I  If  you  are  unable  to  evaluate) 


APPROVED  BY 


K>» 


□□ 


CODING  SIMPLICITY  MEASURE 


Fora  Code:  TeIM.3 


DATA  COLLECTION 


stive  Boolean  or 
ressions  (106) 


Boolean 


1.1  Number  of  negative  or  complicated 
coegttund  Boolean  expressions. 

1.2  Number  of  lines  excluding  foments 
Score-1-  <fiTi  +  irrii 

2.0  Stateeent  Labels  (108) 

2.1  Number  of  statement  labels. 

(Do  not  coast  format  statements) 

2.2  Nuriter  of  lines  excluding  amenta 
(Executable  Statements) 

Score-lr  <H3  +  EE) 

3.0  Nesting  Level  (109) 

3.1  Maximum  nesting  level? 

Score  -  1  +  EE 

4.0  Conditional  Branches  (110) 

4.1  Number  of  conditional  brandies. 

4.2  Nuber  of  executable  statements. 

Score-1-  (ED  [71]) 


□ 

□ 


PREPARED  BY 
DATE: _ 


APPROVED  BY: 
DATE: _ _ 


POPING  SIMPLICITY  MEASURE  (Continued) 

Fora  Code:  TeBM.3 


CYCLE  PHASE:  SOURCE (S) 

IMPLBENTATICN 


8.0  Flow  Top  to  Bottom  (IIS} 

8.1  Is  flow  top  to  bottom? 

(There  should  not  be  any 
backward  branching  GOTDs.) 

Yes  -  1,  No  -  0 
9.0  Local  Variables  (128) 

9.1  Number  of  local  variables. 

9.2  Nmber  of  variables  (local  and 
global) 

Score  -  ED  ♦  El 


II.  METRIC  WORXSECnON: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10)  (Jf  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 
□ATE: _ 


Te-41 


□□ 


MODULAR  IMPLEMENTATION  MEASURE 

Fora  Code: 


TelMO 


CYOI  PHASE: 

IMPLBffiNTATIQN 


SYSTEM  me 


|S0QRffi(S): 


DATA  COLLECTION  W3RJ5BCriON: 


1.1  Number  of  ing  sequence  perm- 

meters  that  are  control  variables. 

1.2  Nuaber  of  calling  sequence  para¬ 
meters.  .  _ 

Score  ■  ll.ll  +  1 1.2l 

2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  •  1,  No  •  0 

3.0  Controlling  Output  (89) 

3.1  Is  output  data  passed  back  to 

calling  module? 

Yes  -  1,  No  -  0 

4.0  Controlling  Return  (90) 

4.1  Is  control  returned  to  calling 
nodule? 

Yes  -  1,  No  -  0 


Metric  Value  • 


Sum  of  Above  Scores 


PREPARED  BY: 
DATE: _ 


APPROVED  BY:a 
DA2E: _ _ 


Te-42 


□□ 


Form  Code:  TeBM.5 


SCORE 


Quantity  of  Cements  (117) 

1.1  Number  of  non -blank  lines  of 

1.2  Number  of  non-blank  lines. 

Score  •  1 1.1 1  +  ll.2l 


■  B 


II.  METRIC  NORKSECTIQN: 


Metric  Value  * 


Sum  of  Above  Scores 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 
DATE: 


Te-43 


EFFECTIVENESS  OF  COMMENTS  MEASURE 

Pom  Code:  TeIMM.6 

LIFE  CYCLE  PHASE: 

IMPLEMENTATICN 

SOURCE(S) :  ' 

□□□ 

NAME: 

I.  DATA  COLLECTION  WCRKSECTION: 

1.0  Prologue  Comments  (118) 

1.1  Are  there  prologue  commits  cantun¬ 
ing  information  about  the  function, 

author,  version  number,  date  inputs,  r-r — 

outputs,  arsuaptions  and  limitations?  Y  N 

Yes  -  1,  No  -  0 

2.0  Control  and  Destinations  Cement  (119) 

2.1  How  many  decision  points  and  transfers  i - 1 

of  control  are  not  cemented?  1  1 

2.2  Total  number  of  decision  points.  1  1 

Score*!- ( 1 2.1 1  +  1 2.2  b 

3.0  Machine  Dependent  Code  Cement  (120) 

3.1  Is  all  machine  language  code  i—t— i 

cemented?  Y  N 

Yes  *  1,  No  •  0 

4.0  Non-standard  HOL  Statements  Comment  (121) 

4.1  Are  non-standard  HOL  statements  Y  N 

Consented? 

Yes  •  1,  No  ■  0 

SCORE 

□ 

□ 

□ 

□ 

PREPARE)  BY: _  APPROVE)  BY: 


Te-44 


EFFECTIVENESS  OF  COMMENTS 


CYCLE  PHASE: 

IMPLEMENTATION 


DATA  COLLECTION  NORKSECTION  (cant 

5.0  Declared  Variables  fomented  (122) 

5.1  How  many  declared  variables  are  not 
described  by  cements? 

5.2  Number  of  variables.. 


Score-1 -  cED  ♦  E3 


6.0  foments  Which  Do  Not  Only  Repeat  the 
Operation  (lZ4) 

6.1  Do  the  commits  do  more  than  repeat 
the  operation? 

Yes  -  1,  No  -  0 

7.0  Identification  of  Cements  (137) 

7.1  Are  commits  set  off  from  the  code 


in  a  uniform  manner? 
Yes  -  1,  No  -  0 


METRIC  WRXSECTION: 


Metric  Value  ■ 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (Jf  If  you  are  unable  to  evaluate) 


PREPARED  BY:a 
DATE: _ 


APPROVED  BY: 

m 

DATE: _ 


Te-45 


□□ 


DESCRIPTIVENESS  OF  IMPLEMENTATION  LANGUAGE  MEASURE 


Fora  Code:  TeBM.7 


atCLZ  PHASE: 

IMPLEMENTATION 


|SOURCE(S): 


data  mri3»cnCN  CRXSECTIQN: 


1.1  Is  high  order  language  used? 
Yes  -  1,  No  »  0 
2.0  Variable  Naees  (123) 


2.1  Are  variable  nine  (anenmics) 
descriptive  of  the  pnysical  or 
ftmrtimii  property  they  represent? 


Yes  •  1,  No  •  0 
3.0  Blocked  and  Indented 


Code  (125) 


3.1  is  the  code  logically  blocked  and 
indented? 


Yes  ■  1,  No  ■  0 


□□ 


DESCRIPTIVENESS  OF  IMPLEMENTATION  LANGUAGE 


Form  Coda:  TelMM. 7 


CYCLE  PHASE: 

IMPLEMENTATION 


SOtSCECS) : 


DATA  COLLECTION  W3RXSECTICN  (continued 
4.0  One  Statement  Per  Line  (126) 

4.1  Haber  of  lines  with  acre  than 
one  statement. 

4.2  Number  of  continuous  lines. 

4.3  Total  number  of  lines  in  a 

module.  -  - 


Score  »  1  - 


liiiJ 

♦ 

4.3 

II.  METRIC  WORKSECTION: 


III.  EVALUATICN  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  Eased  on  the  data  elenents  above? 
(1-10)  Of  If  you  are  unable  to  evaluate) 


Metric  Value  • 


Sum  of  Above  Scores 


WEPARED  BY:. 
DATE: _ 


APPROVED  BY:. 
DfOB: _ 


Te-47 


SIMPLICITY 


JFE  CYCLE  PHASE:  SOURCE  (S) : 

IMPLEMENTATIC^J  TeIM*.l 

tJ  SYSTEM  NMC: 

□  SUBSYSIB!  - - -  - 

□  MDDULE  - 

I.  METRIC  SLAMARY  HDRXSECHON: 

1.  Design  Structure  Measure 

2.  Complexity  Measure 

3.  Coding  Simplicity  Measure 


Fora  Code:  TcDC.l 


TeBW.2 


TeBW.3 


II.  CRITERIA  WCRXSgCTICN: 

Criteria  Value*  Smns  I  I 

No.  of  Metrics 

III.  EVALUATION  HBUBECTICN:  Mutt  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (|  if  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY: 

m 

DATE: _ 


Te-48 


ODD 


r  oo 


SELF-DESCRIPTIVENESS 


Fora  Coda:  TeDC.3 


SLUE  CYCLE  PHASE: 


|  Q  SYSTBt 
SQBSYSTB4 
MuDULE 


IMPLBiENTATICN 


NAME: 


SOURCE  (S) : 

TeDM.5 


TeBM.6 


TeDW.7 


METRIC  SIMttRY  HORKSECTION: 

1.  Quantity  of  Ccomnts  Measure 

2.  Effectiveness  of  Coanents  Measure 

3.  Descriptiveness  of  lapleaentation  Language 
Measure 


SCORE 


II.  CRITERIA  WORKSBCTION: 


.  .  Su®  of  Above  Scores 

Criteria  Value  TfeV  of  Metrics - 


□ 


III-  EVALUATION  MORKSECnCK:  What  is  your  evaluation  of  the 
renewed  pricUcts  based  on  the  Metric s  above? 

(1-10) _  (f  If  you  are  unable  to  evaluate) 


PREPARED  BY: 


APPROVED  BY: 


□  □□ 


□□□ 


N  N  N 


s  |s  |s 


■  7 
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■ 

■ 

■ 
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■ 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


Po-i 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

PORTABILITY  MODULE 


INTRODUCTION  TO  PORTABILITY 

The  Quality  Factor  Portability  represents  a  measure  of  the 
effort  required  to  transfer  a  program  from  one  hardware  con¬ 
figuration  and/or  software  system  environment  to  another. 

FIGURES  Po  -  1  through  Po  -  4  represent  the  hierarchy  of  Metric, 
Criteria  and  Quality  Factor  worksheets  associated  with  Porta¬ 
bility  in  Requirements  Analysis,  Preliminary  Design,  Detailed 
Design  and  Implementation  phases  of  the  Software  Development 
Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORXSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Portability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


PORTABILITY 


QUALITY  FACTOR 


MODULAR 

IMPLBCNEAXICN 

MEASURE 


MEASURE 


CRITERIA 


METRIC 


FIGURE  Po-1 

DETAILED  DESIGN  WORKSHEET  HIERARCHY 


PORTABILITY 


QUALITY  FACTOR 


CRITERIA 


QUALITY 

OP 
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MEASURE 


EFFBCITVHBSS 

OP 
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MEASURE 


OF 

MPLBORXnON 

LANGUAGE 

MEASURE 


FIGURE  Po-2 


STEPS  FOR  COMPLETING  WORKSHEETS 


Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block -Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block -On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0 . 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element. 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


Po-5 


f ‘  v- 


(5.2) 

(5.3) 

, (5.4) 

(5.5) 

(5.6) 

Step  6  - 
(6.1) 

(6.2) 

(6.3) 

(6.4) 

(6.5) 

(6.6) 


Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

FACTOR  WORKSHEETS 

Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


T 


» 


1 


STEPS  FOR  UPDATE  APPLICATIONS 
Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 
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HOW  TO  USE SCORE  CHART  SETS 


When  you  apply  worksheets  to  system/ subsystem/module , 
you  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  -locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  •  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 
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Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 
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Step  4  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart’s  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


TRADE 

OFF 

STUDY 


INITIAL  OR 
AUTHENTICATED 
SYSTEM 


DRAFT 

PRODUCT  |  ACTION 
HB4 


SET 
TO  BE 
APPLIH) 


REQUIRBffiNTS 

ANALYSIS 


PRODUCTS  FOR  ANALYSIS  PHASE 


Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l  -  Trade-off  Study  Reports 

•  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  4  Segment  Specification  (if  any) 

e  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  -  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 
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RNC- 
TICNAL 
PRODUCTS  FLOW 
CHARTS 


DATA  ALGO- 
FOFMAT  RITTW 
ISSC.  DESC. 


SET 
TO  BE 
APPLIED 


SYSTB4 

CPCI 
TEST 

TEST  |  PRO- 
PLANS 


PRELIMINARY 


ESP  PRODUCTS  FOR  DESIGN  PHASE 

Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 

•  PToduct  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

•  Product  B.  Preliminary  Product  Specifications, 

including  the  above 

•  Product  D.  CDR  minutes  I  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 


NOTE:  If  these  products  do  not  contain  the  information 

requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


ALTERED  PRODUCT  SPECS 


DETAILED  DESIQ* 
WORKSHEETS  (UPDATE) 


ESP  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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APPLIED 


PRELIMINARY  DESIGN 
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WJklbmTS 
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ESP  PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 


Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 


A 

•  Product  1:  Final  CPCI  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system-level 
Test  Procedures 

B-D 

e  Product  l:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-d 

e  Product  2:  Computer  Program  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 
B-D 

e  "Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changea. 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


MODULAR  IMPLEMENTATION  MEASURE 


Fora  Code:  PoDEM.l 


[LIFE  CYCLE  PHASE: 

DETAIL  DESICN 


□  SYSIBi 
SUBSYSIBC 
MODULE 


NAME: 


SOURCE  (S) : 


I. 


DATA  COLLECTION  NORKSECTIOH: 


1-0  Ccntrolling  Parametersf  87  ) 

1.1  Nuaber  of  calling  sequence  para¬ 
meters  that  are  control  variables. 

1.2  Nimber  of  calling  sequence  para¬ 
meters. 

Score  -  fl.l|  - 

2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  ■  1,  No  •  0 

3.0  Controlling  Output  (  89) 

3.1  Is  output  data  passed  back  to 
calling  module? 

Yes  -  1,  No  -  0 

4.0  Controlling  Return  (  90 ) 

4.1  Is  control  returned  to  calling 
module? 


1.2 


Yes  ■  1,  No  -0 


MODULAR  IMPLEMENTATION  MEASURE  (Contirued 


For*  Coda 


continued! : 


other  nodules? 
Yes  -  1,  No  -  0 


storage  independent  of 


II.  METRIC  -'rwCSECnON: 


Metric  Value 


Sum  of  Above  Scores  I  I 

■  IK"1  l . ) 


III.  EVALUATION  NORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elenents  above? 
(1-10) _ (3  If  you  are  unable  to  evaluate) 


□□ 


□□ 


idence  on  Software  System 


1.1  Nuaber  of  references  to  system 
library  routines,  utilities,  or 
other  system  provided  facilities 
or  functions. 

1.2  Nxmber  of  total  lines  of  code. 
Score-  l-(  U1  t  JL2] ) 

Standard  Language  Used  (96) 

2.1  Is  a  caencn  standard  subset  of  a 
programing  language  to  be  used? 

Yes  -  l,  No  •  0 


II.  METRIC  WDRKSECTION: 


Metric  Value 


Sum  of  Above  Scores 


III.  EVALUATION 
(1-10) _ 


ON:  Mutt  is  your  evaluation  of  the 
sed  on  the  data  eleamts  above? 

(0  If  you  are  unable  to  evaluate) 
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SOFTWARE  SYSTEM  INDEP 


Form  Code :  PoDDC.3 


CRITERIA  WORX5ECTION: 


. .  Sun  of  Above  Score 

Criteria  Value  •JBT^FTJeEici - 


III.  EVALUATION  WCMSBCTICN;  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


-21 


PORTABILITY 


Porn  Cods:  PoDDF.l 


CYCLE  PHASE: 

DETAIL  VESlGi 


SYSTB4  W)C 


SOURCE  (S) : 
PoDDC.l 


PoDDC.2 


CRITERIA  gMWOf  WCRKSECTICN 

1.  Modularity 

2.  Machine  Independence 

3.  Software  System  Independence 


Factor  Value  • 


Sum  of  Above  Scores 


III.  EVALUATION  WCMBBCTION:  Wist  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 


□  □□ 


PORTABILITY 

Complete  this  score  chart  if  application  of  this  set  of  Worksheets  is  appropri¬ 
ate  when  products  of  Al,  A2,  A3,  A4,  B  and  D  in  Design  Phase  are  available. 


Answer  of 
Data 
Element 


Data 

Element 

Score 


Metric 

Score 


Criteria 

Score 


Factor 

Score 


FIRST  MEASUREMENT 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  C  in  Coding  and  Checkout  phase  is  available. 


Ntnfeer 


Answer  of 

Data 

H 


Score 


Metric  Criteria  Factor 

Score  Score  Score 


□□ 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


EFFECTIVENESS  OF  CCM4ENTS  MEASURE 

Form  Code:  PoBM.3 


SOURCE (S) : 


DATA  COLLECTION  VCRXSECTION: 
1.0  Prologue  Comments  (118) 


1.1  Are  there  prologue  eomaents  contain  - 
ning  information  about  the  function, 
author,  version  number,  date  inputs, 
outputs,  assunptions  and  limitations? 

Yes  -  1,  No  »  0 

2.0  Control  and  Destinations  Comment  (119) 

2.1  How  many  decision  points  and  trans¬ 
fers  of  control  are  not  ccammntad? 

2.2  Total  amber  of  decision  points. 
Score-1- (  1 2.1 1  ♦  [2.2 1 1 


3.0  Machine 


at  Code 


3.1  Is  all  machine  language  code 


Yes  •  1,  No  ■  0 

4.0  Non-standard  HOL  Statements  Comment  ( 
4.1  Are  non-standard  HX  statements 


Yes  •  1,  No 


EFFECTIVENESS  OF  OCmENTS  MEASURE  (Continued) 

Fora  Code:  PoDM.3 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


SY5TB4 


DATA  COLLECTICN 


5.0  Declared  Variables  Cogmnted  (122) 

5.1  How  many  declared  variables  are  not 
described  by  comments? 

5.2  Number  of  variables. 

Score-1- (  l5.1 1  +  S.2  ) 


6.0  Comments  Much  Do  Not 


st  the 


6.1  Do  the  coements  do  more  than  repeat 
the  operation? 

Yes  -  1,  No  •  0 

7.0  Identification  of  Comments  (137) 

7.1  Are  garments  set  off  from  the  code 
in  a  uniform  meaner? 

Yes  •  1,  No  -  0 


II.  METRIC  WORKSECTTON: 


Metric  Value  > 


Sum  of  Above  Scores 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  data  aliments  above? 

( 9  If  you  are  unable  to  evaluate) 


DESCRIPTIVENESS  OF  IMPLEMENTATION  LANGUAGE  MEASURE 


Form  Code:  PoD>M.4 


LIFE  CYCLE  PHASE: 

IMPLSMENTATI CN 


□  SYSTEM 

□  SUBSYSTEM 

□  MODULE 


DATA  COLLECTION  WORX5ECTION 


1.0  High  Order  i (107)  _____ 

1.1  Is  high  order  language  needed?  IyIn 

Yes  -  1,  No  -  0 

2.0  Variable  Names  (123) 

2.1  Are  variable  nane  (anemonices) 

descriptive  of  the  physical  or  fun-  i— r— 

ctional  property  they  represent?  |Y|N 

Yes  -  1,  No  -  0 

3.0  Blocked  and  Indented  Source  Code  (125) 

3.1  Is  the  code  logically  blocked  and 
indented? 

Yes  •  1,  No  •  0  0N 


MACHINE  INDEPENDENCE  MEASURE 


Form  Code:  PoIW.S 


'LIFE  CYCLE  PHASE: 

IMPLB1ENTATION 


□  SYSIB* 
SUBSYSTB4 
MODULE 


NA>€: 


SOURCE (S) : 


I.  DATA  COLLECTION  HCRKSECHON: 

1.0  Pete  Representation  Machine  Independent  (132) 


SCORE 


II. 


III. 


IV. 


1.1  Is  date  representation  machine 
independent? 

Yes  -  1,  No  ■  0 


METRIC  WORXSECHON: 

Metric  Value 


San  of  Above  Scores 
No.  of  Data  Elements 


EVALUATION  pTMtsfirTTCM:  feat  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elownts  above? 
(1-10)  (0  If  you  are  unable  to  evaluate) 


INSPECTOR’S  CBMEMg: 


PREPARED 


APPROVED  BY: 


DATE: 


DA3E: 


Po-31 


BY 


SOFTWARE  SYSTEM  INDEPENDENCE  MEASURE 

Foot  Code: 


PoD-M.6 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


XVWCi 


on  Software 


Utilit 


1.1  Number  of  references  to  system 
library  routines,  utilities,  at 
other  system  provided  facilities 
or  functions. 

1.2  Nnber  of  total  lines  of  code. 

Score-1*  dlTI  +  O.) 


II.  METRIC  WORKSECHON: 


Metric  Value 


Sun  of  Above  Scores 
No.  of  bata  Elements 


III.  EVALUATION 
Cl-10) _ 


What  is  your  evaluation  of  the 
on  the  data  elements  above? 

{ 9  If  you  are  unable  to  evaluate) 


MODULARITY 

Fora  Code:  PoIMC.l 

LIFE  CYCLE  PHASE: 

IMPLANTATION 

SQURCE(S) : 

POM4.1 

NAME: 

I.  METRIC  SUMMARY  WORKSECTION: 

1.  Modular  Iaplementation  Measure 

SCORE 

□ 

II.  CRITERIA  WORKSECTION: 

□ 

III.  EVALUATION  WCRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  aetrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 

PREPARED  BY: 


APPROVED  BY: 


□o 


SELF-DESCRIPTIVENESS 

Fora  Code:_ 

PoIMC.2 

ILIFE  CYCLE  PHASE: 

IMPLS4ENTAIICN 


□  SYSTEM 
SUBSYSTEM 
MODULE 


UUA. 
nrPE  • 


SOURCE (S^ : 
PoBM.2 


PoDfd.3 


PoBM.4 


I.  METRIC  SUMMARY  WQRXSECTICN: 

1 .  Quantity  of  Coonents  Measure 

2.  Effectiveness  of  Comments  Measure 

3.  Descriptiveness  of  Implementation  Language 
Measure 


SPORE 


II.  QUTERIA  WORXSECTION: 


o_.- — - Sun  of  Above  Scorn 

Criteria  Value  ^  pTHetHcs - * 


□ 


III. 


EVALUATION  WCMBECTICN:  What  is  your  evaluation  of  the 
raviei^  products  based  on  the  aetrics  above? 

(1*10)  (I  If  you  are  unable  to  evaluate) 


□  □□ 


□□ 


MACHINE  INDEPENDENCE 


Fora  Code:  PoIMC.3 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 

SOURCE  (S) : 

PoIMM.S 

□□□ 

NAME: 

I.  METRIC  SUMMARY  WRKSECTICN: 

1.  Machine  Independence  Measure 

SCORE 

□ 

II.  CRITERIA  WORKSECTION: 

.  ..  .  Sum  of  Above  Scores 

Criteria  Value  Tfe.  0f  Metrics 

□ 

III.  EVALUATICN  WORKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 

CYCLE  PHASE: 

IMPLEMENTATION 


SYSIBf 


SOFTWARE  SYSTEM  INDEPENDENCE 

Fora  Code 


SOURCE (S): 
PoBM.6 


METRIC  SUGARY  WORKSECTICN; 

1.  Software  System  Independence  Measure 


(1-10) 


ON:  What  is  your  evaluation  of  the 
Lead  on  the  metrics  above? 

(|  If  you  are  unable  to  evaluate) 


»  < 


•  v 


□□ 


□□□□ 


PORTABILITY 


Complete  this  score  chart  if  application  of  this  set  of  worksheets  is  appropri 
ate  when  products  of  A  and  B  in  Coding  and  Checkout  Phase  are  available. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Ntnber  w— w  Score  Score  Score  Score 


PORTABILITY 


Conplete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro* 
priate  when  products  of  B-D2  in  Test  and  Integration  Phase  is  available. 

Data  Answer  of  Data 

ELeaant  Data  Elegant  Metric  Criteria  Factor 

Nuaber  Element  Score  Score  Score  Score 


PORTABILITY 


Conplete  this  score  chart  if  application  of  this  set  of  worksheets  is  appro* 
priatc  when  products  of  B-D2  in  Test  and  Integration  Phase  is  available. 

Data  .  Answer  of  Beta 

Eleatnt  Data  ELenent  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 
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This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
modules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

REUSABILITY  MODULE 


INTRODUCTION  TO  REUSABILITY 

The  Quality  Factor  Reusability  represents  a  measure  of  the 
extent  to  which  a  program  can  be  used  in  other  applications  - 
related  to  the  packaging  and  scope  of  the  functions  that  pro¬ 
grams  perform.  FIGURES  Ru  -  1  through  Ru  -  4  represent  the 
hierarchy  of  Metric,  Criteria  and  Quality  Factor  worksheets 
associated  with  Reusability  in  Requirements  Analysis,  Prelimin¬ 
ary  Design,  Detailed  Design  and  Implementation  phases  of  the 
Software  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  BSD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Reusability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


i 


■TMf**-- 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block- Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block-On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  Requirements  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 

Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans¬ 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 

If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element . 

(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 

Step  5  -  CRITERIA  WORKSHEETS 

(5.1)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 


(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 


(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value: 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


Ru-6 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements”,  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets”,  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


HOW  TO  USE  SCORE  CHART  SETS 

When  you  apply  worksheets  to  system/subsystem/module, 
y.oa  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores,, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  §  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEAS  REMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  ■  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 

(1.1)  Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

(1.2)  Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

(1.3)  Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

(1.4)  Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 

Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Worksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 


Step  4  -  FACTOR  SCORE 

Each  box  in  this  column  represents  the  score 
from  the  Factor  Worksection  of  a  Quality  Factor 
Worksheet.  Transfer  the  Factor  from  the  set's 
Factor  Worksheet  into  corresponding  boxes  of  the 
Score  Chart's  Factor  Score  Column. 

These  Score  Chart  sets  will  provide  a  historical 
base  for  repeated  application  of  the  metrics  to  a  system 
at  all  levels.  They  are  also  a  useful  means  of  reporting 
your  finds  to  the  program  manager  in  a  condensed  and  com¬ 
pact  form. 


« 
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PRODUCTS 


TRADE¬ 

OFF 

STUDY 


SET 
TO  BE 
APPLIED 


INITIAL  OR 
AUTHENTICATED 

AUTHENTICATED 

m 

srsrm 

DRAFT 

SPEC, 

SPEC 

LEVEL 

PRODUCT 

M  ''  ii 

FOR 

SPEC 

SPECS 

SPEC 

CPCI 

MINUTES 

AND 

ACTION 

rrm 

RESPONSES 


RBQUIRBENT5 

ANALYSIS 


ESP  PRODUCTS  FOR  ANALYSIS  PHASE 

Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  5  Segment  Specification  (if  any) 

e  '  Product  B.l  -  Authenticated  Development  Specification  foT 

each  CPCI 

e  Product  B.2  -  Possible  higher-level  specification,  ICD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Part3  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  and  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor-supplied 
information  should  be  searched. 
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CDR 


FUNC¬ 

TIONAL 

n 

DATA 

ALGO- 

PRELIMI¬ 

NARY 

mm 

systbJ 

PRELIM. 

PRODUCTS 

FLOW 

FLOW 

FORMAT 

RITTM 

CPCI 

CHARTS 

CHART 

DESC. 

TEST 

■ 

■ 

TEST 

PLANS 

PRO¬ 

CEDURES 

WORKSffiE 
SET 
TO  BE 
APPLIED 


PRELIMINARY 

DESIGN 

WORKSHEETS, 

(UPDATE) 


ESD  PRODUCTS  FOR  DESICH  PHASE 


DETAILED  DESIQt 
WORKSHEETS 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 


Product  A.l: 
Product  A. 2: 
Product  A. 3: 
Product  A. 4: 

Product  B.  : 

Product  D .  : 


Functional  flowcharts 

Detailed  flowcharts 

Data  format  descriptions 

Descriptions  of  algorithms  not 
previously  prescribed 

Preliminary  Product  Specifications, 
including  the  above 

CDR  minutes  4  action  item  responses 


Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note. 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


SET  TO  BE 
APPLIED 


ALTERED  PRODUCT  SPECS 


)  DESIQf 
(UPDATE) 


ESD  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 


Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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PRODUCTS 


PINAL 

CPQ 


PRCCE- 


srcisf 


MODIFIED 
CODING  PRODUCT  I  LEVEL 


SET  TO 
APPLIED 


TATICN 


PRODUCTS  FOR  TEST  AND 
INTEGRATION  PHASE 

Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCI  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system- level 
Test  Procedures 

B-D 

e  “Product  1:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  Product  2:  Computer  Progrpm  Coding  Changes 

Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  “Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I CD  and  changea. 


NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 


□□I 


LIFE  CYCLE  PHASE: 

DETAIL  DESIC34 


GENERALITY  CHECKLIST 


SOURCE (S) : 


Fora  Code:  RuDEM.2 


DATA  COLLECTION  WJRKSECTION: 

1.0 


1.1  Does  the  module  not  mix  input, 
output,  and  processing  functions 
in  the  same  module? 

Yes  -  1,  No  •  0 

2.0  Machine  Dependent  Functions  [93) 

2.1  Nimber  of  machine  dependent  functions 
performed. 

Score  ■  1  +  S3 
3.0  Unlimited  Data  Volume  (94) 

3.1  Is  processing  data  volume  unlimited  ? 
Yes  -  1,  No  -  0 

4.0  Unlimited  Data  Value  (95) 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
an  the  data  elements  above? 

(0  If  you  are  unable  to  evaluate) 


MACHINE  INDEPENDENCE  MEASURE 


Fora  Code:  RuDEM.3 


1.1  Is  the  programing  language  availa¬ 
ble  in  other  aachines? 

Yes  •  1,  No  •  0 


II.  METRIC  WORKSECTICN: 


Metric  Value  -  j| 


III.  EVALUATION  WDRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products'  based  on  the  data  elcemts  above? 
(1-10) _ 0>  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
DATE: _ 


APPROVED  BY: 
DA2E: _ _ 
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SOFTWARE  SYSTEM  INDEPENDENCE  MEASURE 

Fora  Code:  RuEEM.4 


LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 


SCIIO(S) : 


□  SYSTB4 
f*l  SUBSYSTEM 

□  MODULE 


NAME: 


DATA  COLLECTION  WCRK5ECTI0N: 


SCORE 


1.1  Nuober  of  references  to  system 
library  routines,  utilities ,  or 
other  system  provided  facilities 
or  functions. 

1.2  Nuober  of  total  lines  of  code. 
Score  -  1  -  dim  4-  (ITT) 


2.1  Is  a  cowan  standard  subset  of  a 
programing  language  to  be  used? 

Yes  •  1,  No  -  0 


II.  METRIC  WORKSECTION: 


Metric  Value  ,  Sum  of  Above  Scores 


III.  EVALUATION  WORKSECTIGN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1*10)  (j)  If  you  are  unable  to  evaluate) 

—  INSPECTOR'S  COMMENTS: 


PREPARED  BY: 


APPROVED  BY: 
DATE: _ 
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□□ 


-  »•  — 


GENERALITY 

Fonn  Code:  RuDDC.2  _  _ . 

LIFE  CYCLE  PHASE: 

DETAIL  DESIGN 

SCURQs(S) : 

RuDCM.2 

□  SYSIB1 

□  SUBSYSTB* 

□  HOODIE 

mie . 

I.  METRIC  SUMMARY  WORKSECTION: 

1.  Generality  Qiecklist 

SCORE 

□ 

II.  CRITERIA  WORKSECTION:  Sun  of  Above 

Criteria  Value  ■j^TTStHci 

□ 

III.  EVALUATION  WORKSECTION:  Mutt  is  your  evaluation  of  the 
reviewed  products  based  on  the  settles  above? 

(1-10)  (|  if  you  axe  unable  to  evaluate) 

PREPARED  «T:  APPROVED  BY: 


DATE: 


QMS: 
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SOFTWARE  SYSTEM  INDEPENDENCE 


Fora  Code:  Rnirof-A 


LIFE  CYCLE  PHASE: 

DETAIL  DESIQJ 

SOURCE  (S): 

RuDEM.4 

□  SYSTEM 

□  SUBSYSIB4 

□  MODULE 

NAME: 

I.  METRIC  SUMMARY  WORKSECTION: 


II. 


III. 


1.  Software  System  Independence  Measure 


CRITERIA  WORKSECTION: 


Sum  of  Above 


Criteria  Value  ■ 


Scores 

No.  of  Metrics 


EVALUATION  WORKSECTION:  feat  is  your  evaluation  of  the 
reviewed  products  based  on  the  metrics  above? 

(1-10) _  (jl  If  you  are  unable  to  evaluate) 


REUSABILITY 


CYCLE  PHASE: 
DETAIL  DESKS') 


CRITERIA  SUMMARY  WORXSECTION: 


1.  Modularity 

2.  Generality 

3.  Machine  Independence 

4.  Software  System  Independence 


Form  Code:  RuDDF.l 


SOURCE (S) : 
RuDDC.l 

RuDDC.2 

RuDDC.3 

RuDDC.4 

II.  FACTOR  WORKSECriCN: 


Factor  Value 


Sum  of  Above  Scores 


).  of  Criteria 


III.  EVALUATION 
(1-10) _ 


What  is  your  evaluation  of  the 
on  the  criteria  above? 

01  If  you  are  unable  to  evaluate) 


□□□□ 


s?|!5|s8fe|a|s|}s|8|s 


REUSABILITY 


MODULAR  IMPLEMENTATION  MEASURE 

Form  Code: 


LIFE  CYCLE  PHASE: 

IMPLB4ENTATICN 

SCURCE(S): 

□□□ 

NAME: 

I.  DATA  COLLECTION  WCRJSECriON: 

1.0  Controlling  Parameters  (87) 

1.1  Nwber  of  celling  sequence  parameters 
that  ere  control  variables. 


SCORE 


1.2  Number  of  calling  sequence  parameters 

Score  •  [Tl]  t  [T7T 

2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 


If  Yes  -  1,  No  -0 

3.0  Controlling  (Xxtput  (89) 

3.1  Is  output  data  passed  back  to 
calling  module? 

If  Yes  •  1,  No  -0 


N 


□ 

□ 

□ 


□  □ 


□□L 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


systbi  me.: 


GENERALITY  CHECKLIST 


|90URCE(S): 


Fot»  Code: 


DATA  ODLLECTION  WRXSECTION: 

1-0  Unaixed  Processing  end  Input/Output  (S 

1.1  Does  the  nodule  not  mix  input, 
output,  and  processing  functions 
in  the  sene  module? 


Yes  -  1,  No  ■  0 


2.1  Nunfcer  of  machine  dependent  functions 
performed,  _ 


Score  •  1  +  1 2.1 1 
3.°  Unlimited  Data  Volime  (94) 

3.1  Can  the  mounts  of  data  that  can  be 
processed  be  unlimited? 

Yes  -  1,  No  ■  0 

4.0  Unlimited  Data  Value  (95) 

4.1  In  the  Implementation  Riase,  is  the  value 
of  data  that  can  be  processed  unlimited? 

Yes  ■  1,  No  -  0 


II.  METRIC  WORKSECTION: 


Metric  Value  •  gum  of  Above  .Scores 
No.  of  Data  Elements 


III.  EVALUATION  jggSBCnCN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _ (0  If  you  are  unable  to  evaluate) 


CYCLE  PHASE: 
IMPLEMENTATION 


□  SYSTEM 

□  SUBSYSTEM 

□  MXULE 


1JAJJC  • 


Foun  Code: 


SOUO(S): 


SCORE 


1.1  Number  of  non-blank  lines  of 

1.2  Nuober  of  non-blank  lines. 


score  ■  11.11  + 


METRIC  WORKSECTION: 


Metric  Value 


Sun  of  Above  Scores 


III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  an  the  data  elaaents  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


□□ 


EFFECTIVENESS  OF  COMENTS  MEASURE 


Code: 


CYCLE  PHASE: 

IMPLB1ENTATI0N 


SOURCE  (S) : 


DATA  COLLECTION  TCRJSECTION: 

1.0  Prologue  Cements  (118) 

1.1  Are  there  prologue  cements  containing 
information  about  the  function,  author, 
version  number,  date  inputs ,  outputs, 
as  sunlit  ions  and  limitations? 

Yes  •  1,  No'  ■  0 

2.0  Control  and  Destinations  Comment  (119) 

2.1  How  many  decision  points  and  transfers 
of  control  are  not  coanented? 

2.2  Total  number  of  decision  points. 

Score  ■  l  -  cTCT  -4-mi) 

3.0  Mfhine  Dependent  Code  Cement  (120) 

3.1  Is  all  machine  language  ode 
coanented? 

Yes  -  1,  No  •  0 

4.0  Non-standard  HX  Statements  Comment  (121) 

4.1  Are  non-standard  HOL  statements 
coanented? 

Yes  •  1,  No  •  0 


BY 


BY 


EFFECTIVENESS  OF  QSMENrS  MEASURE 

Foot  Code:  RuIW.4 


(LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


I  {^“j  SYSTEM 

□  SDBSYSIB4 

□  MXOUB 


UUC. 

rpnc* 


S0CESI2(S) : 


I. 


DATA  COLLECTION  BCRKSBCTICN  (continued) : 
5.0  Declared  Variables  fncanteri  (122) 

S.l  How  aany  declared  variables  are  not 
described  by  comnents? 


5.2  Nuaber  of  variables. 

Score  -  l  -  f  im-r  TT71) 

6.0  Cgeemts  Which  Do  Not  Only  Repeat  the  Operation.  (124) 


6.1  Do  the 

operation? 


do  sore  than  repeat  the 


YN 


Yes  -  1,  No  -  0 

7.0  Identification  of  Co-eats  (137) 

7.1  Are  coa— nrs  set  off  from  the  code 
in  a  unifora  earner? 

Yes  •  1,  No  ■  0 


YN 


sgore 


□ 


□ 


□ 


II.  METRIC  NCRKSECnON: 


Metric  Value 


Sua  of  Above  Scores 
No.  of  Data  Elements 


□ 


III.  EVALUATION  WCMCSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  baaed  on  the  data  elements  above? 
(1-10) _  (0  If  you  are  unable  to  evaluate) 


IV.  INSPECTOR’S  OCMMENTS: 


PREPARED  BY: 


APPROVED  BY: 


DESCRIPTIVENESS  OF  IMPLB4ENTATIGN  LANGUAGE  MEASURE 

Form  Code:  RuMi.s 

LIFE  CYCLE  PHASE: 

IMPLEMENTATIGM 

SOURCE  (S): 

□  SYSTB-t 

□  SUBSYSTB4 

□  MODULE 

NA*E: 

I.  DATA  COLLECTION  NORKSECTION: 

1.0  High  Order  Language  (107) 

1.1  Is  high  order  language  used?  Y  N 

Yes  -  1,  No  -  0 

2.0  Variables  Names  (123) 

2.1  Are  variable  name  (anaaonices) 

descriptive  of  the  physical  or  r-r- 

functional  property  they  represent?  Y  N 

Yes  »  1,  No  -  0 

3.0  Blocked  and  Indented  Source  Code  (125) 

3.1  Is  the  code  logically  blocked  and  rr— i 

indented?  |Y|N| 

Yes  -  1,  No  -  0 

SCORE 

□ 

□ 

□ 

PREPARED  BY: 
DATE: _ 


APPROVED  BY: 
DAIS: _ 


DESCRIPTIVENESS  OF  IMPLEMENTATION  LANGUAGE  MEASURE 

Form  Code:  RuI^.S 


CYCLE  PHASE: 

IMPLEMENTATION 


DATA  COLLECTION 
4.0  One  Statement  Per  Line  (126) 


4.1  Number  of  lines  with  gore  than 
one  statement. 

4.2  Nimber  of  asxtinucxis  lines. 

4.3  The  total  zunber  of  lines  in  a 
module. 


Score  ■  1  - 


U.i! 

♦ 

S3 

S3 

II.  METRIC  WQRXSECTION: 


Metric  Value  ■ 


Sum  of  Above  Scores 


III.  EVALUATION  WORCSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10)  (0  If  you  are  unable  to  evaluate) 


□  □□ 


□□ 


DO 


SELF-DESCRIPTIVENESS 


Fora  Code:  RuIMC.2 


(LIFE  CYCLE  PHASE: 
IMPLEMENTATION 

(  O  SYSTEM 


MODULE 


SOURCES): 

RuBW.3 


RuttM.4 


RuDtM.S 


I-  >cnuc  SLNOUtr  womsection: 

1.  Quncitr  of  Cn—nri  Measure 

2.  Effectiveness  of  Counts  Measure 

itatiaa  Language 


3. 


Descriptiveness  of  1^)1 
Measure 


II.  CRITERIA  M3RK5ECTTON: 


Sum  of  Above 

VHu.  - 


SCORE 


□ 


III.  EVALUATION  WOWSBCTION:  What  is  your  evaluation  of  the 
reviewed  proiicts  based  on  the  metrics  above? 

(1*10)  (0  If  you  are  unable  to  evaluate) 


PREPARED  BY: 
DATE: _ 


APPROVED  BY: 
DAIS: _ 


Ru  -  38 


□  □□ 


GENERALITY 


Form  Code:  Ru 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 


SOURCE(S): 

RuBM.2 


BY 


BY 


SOFTWARE  SYSIB4  INDEPENDENCE 


Code: 


Tona 


RuIMC.4 


[LIFE  GfCLE  PHASE: 
IMPLEMENTATION 


□  SYSTEM 

□  SUBSYSIB1 

□  MODULE 


mz.: 


SOURCE  (S) : 
RuDM.6 


SCORE 


I.  METRIC  SUMMARY  WORXSECTTON: 

1.  Software  System  Independence  Measure 


□ 


II. 


III. 


CRITERIA  VORKSECTION: 


Sum  of  Above 


Criteria  Value 


Scores 

No.  of  Metrics 


EVALUATION  MQgXSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  Eased  on  the  metrics  above? 

(1-10)  (fi  if  you  are  unable  to  evaluate) 


□o 


REUSABILITY 


linfgd 


CYCLE  PHASE: 

IMPLEMENTATION 


SOURCE  (S) : 

RuIMC.l 

RaIMC.2 

RuIMC.3 

RuIMC.4 

CRITERIA  SUMAKY  WORXSECTION: 

1.  Modularity 

2.  Generality 

3.  Self-Descriptiveness 

4.  Software  System  Independence 


II.  FACTOR  NORXSECTION: 


Factor  Value  ■ 


Sum  of  Above  Scores 


III.  EVALUATION 
(1-10) _ 


What  is  you r  evaluation  of  the 
on  the  criteria  above? 

(jj  If  you  are  unable  to  evaluate) 


BY 


BY 


□□□□ 


■RBfiftB&ni 

Cooplete  this  score  chart  if  application  of  this  set  of  "fj?1***  **  «**r£>* 
prSte  when  products  of  B-D2  in  Test  an.  Integration  Fhsse  is  available. 


i  Date  Answer  of  Pets 
I  nSbt  BLeSmt  TScoreT 


Metric  Criteria  Factor 

Score  Score  Score 


Date 


-  •  7*«,-  v v-  r  V-rflVl-**-. t-WI—l 0¥>1I^  1  '«  .'vw 
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TITLE _ PAGE 
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WORKSHEETS  FOR  IMPLEMENTATION  PHASE . Ip- 35 


This  QUALITY  FACTOR  MODULE  is  one  of  a  set  of  eleven 
nodules  that  are  components  of  the  "COMPUTER  SYSTEMS 
ACQUISITION  METRICS  HANDBOOK". 


Ip-i 


INTRODUCTION  AND  INSTRUCTIONS 
FOR 

INTEROPERABILITY  MODULE 


INTRODUCTION  TO  INTEROPERABILITY 

The  Quality  Factor  Interoperability  represents  a  measure 
of  the  effort  required  to  couple  one  system  with  another. 
FIGURES  Ip  -  1  through  Ip  -  4  represent  the  hierarchy  of 
Metric,  Criteria  and  Quality  Factor  worksheets  associated 
with  Interoperability  in  Requirements  Analysis,  Preliminary 
Design,  Detailed  Design  and  Implementation  phases  of  the 
Software  Development  Life  Cycle. 

The  worksheets  in  this  module  are  grouped  into  four  sep¬ 
arate  sets,  one  set  for  each  Life  Cycle  phase.  Each  set  con¬ 
sists  of  Metric  worksheets.  Criteria  worksheets,  a  Factor 
worksheet  and  score  charts. 

METRIC  WORKSHEETS 

One  or  more  DATA  COLLECTION  WORKSECTIONS  which  contain 
the  data  element  scores  for  each  Metric,  and  each  Met¬ 
ric's  Final  Score. 

CRITERIA  WORKSHEETS 

One  or  more  METRIC  SUMMARY  WORKSECTIONS  which  contain 
the  Metrics  scores  for  each  Criteria,  and  each  Criteria's 
Final  Score. 

FACTOR  WORKSHEETS 

One  CRITERIA  SUMMARY  WORKSECTION  which  contains  each 
Criteria's  score,  and  the  Final  Score  for  the  Factor  for 
that  phase. 

SCORE  CHARTS: 

One  or  more  charts  that  record  worksheet  values  through¬ 
out  the  Life  Cycle. 


Ip-1 


•-.  I1--*" 


However,  the  sequence  in  which  the  worksheet  sets  are  applied 
is  dependent  on  the  availability  of  ESD  Computer  Program  Life 
Cycle  products.  Usually  the  worksheets  will  be  applied 
sequentially,  except  for  updates. 

These  instructions  show  when  the  Interoperability  work¬ 
sheet  sets  and  Score  Charts  are  applied  during  the  Computer 
Program  Life  Cycle.  The  Product  Code  referred  to  on  the 
Score  Charts  is  the  code  defined  in  the  ESD  Guidebook  Series, 
Software  Acquisition  Management  Guidebook,  Life  Cycle  Events 
and  reillustrated  in  the  General  Instructions.  The  Product 
Tables  map  the  ESD  Computer  Program  Life  Cycle  Products  to 
the  Metric  Worksheets.  The  balance  of  these  instructions  are 
step-by-step  procedures  for  completing  the  worksheets  and 
score  chart  summaries. 


INTEROPERABILITY 


QUALITY  FACTOR 


jr  .  -...-v-  - 


^ — 

— 

COMMUNICATIONS 

COMMONALITY 

DATA 

COMMONALITY 

COMMUNICATIONS 

COMMONALITY 

CHECKLIST 

DATA 

COMMONALITY 

CHECKLIST 

CRITERIA 


METRIC 


FIGURE  Ip-1. 

REQUIREMENTS  ANALYSIS  WORKSHEET  HIERARCHY 


FIGURE  Ip-2 

PRELIMINARY  DESIGN  WORKSHEET  HIERARCHY 


P-3 


FIGURE  IP-3 

DETAILED  DESIGN  WORKSHEET  HIERARCHY 


QUALITY  FACTOR 


CRITERIA 


METRIC 


FI6URE  Ip -4 

IMPLEMENTATION  WORKSHEET  HIERARCHY 


STEPS  FOR  COMPLETING  WORKSHEETS 
Step  1 

Enter  the  appropriate  information  in  the  follow¬ 
ing  blocks: 

(1.1)  SYSTEM/SUBSYSTEM/MODULE  block- Indicate  the 
level  of  application. 

(1.2)  NAME  block-List  the  name  of  the  system  being 
measured. 

(1.3)  SOURCE  block-On  Metric  Worksheets,  list  the 
source  document. 

(1.4)  The  PREPARED  BY  and  DATE  blocks  should  con¬ 
tain  the  name  of  the  person  performing  the 
software  metrics,  and  the  current  date. 

Step  2 

Determine  the  current  Computer  Program  Life 
Cycle  Phase.  This  module  is  concerned  with  four  phases 
of  the  Computer  Program  Life  Cycle: 

(2.1)  R.  uireraents  Analysis 

(2.2)  Preliminary  Design 

(2.3)  Detail  Design 

(2.4)  Implementation 
Step  3 

Determine  the  available  products  from  the  phase 
and  which  data  worksheet  set  to  apply.  Each  Life 
Cycle  phase  has  at  least  three  different  activities 
associated  with  it.  The  result  of  the  completion  of  an 
activity  is  one  or  more  documents  called  products . 


Ip-5 


Before  a  worksheet  set  can  be  applied,  specific  pro¬ 
ducts  associated  with  it  must  be  obtained.  The  Pro¬ 
duct  Tables,  at  the  end  of  the  instructions,  detail 
the  products  required  for  each  worksheet  set. 


Step  4  -  METRIC  WORKSHEETS 

(4.1)  Answer  each  question  in  the  Data  Collection 
Worksection  as  it  applies  to  the  system/ 
subsystem/module  you  are  measuring  and  trans 
fer  its  scored  value  to  its  corresponding 
SCORE  block. 


If  the  question  calls  for  a  YES/NO  type 
response,  a  YES  answer  is  scored  at  a  1. 

a  NO  answer  is  scored  as  a  0. 

The  questions  that  call  for  a  numeric 
quantity  type  response  have  specific  in¬ 
structions  on  how  to  score  the  response. 
Follow  the  instructions  under  each  data 
element . 


(4.2)  Find  the  Metric  Value  by  following  the 
algorithm  on  the  worksheet. 

(4.3)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  data  elements  as 
they  apply  to  the  system. 

(4.4)  When  you  have  repeated  this  procedure  for 
all  the  Metric  Worksheets  in  the  set,  you 
are  ready  to  use  the  Criteria  Worksheets. 


Step  S  -  CRITERIA  WORKSHEETS 

(S.l)  Each  numbered  item  in  the  Metric  Summary 

Worksection  represents  an  individual  Metric 
Value. 
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(5.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Criteria  Worksheet. 

(5.3)  Find  the  Metric  Worksheet  with  the  same  Form 
Code  and  transfer  its  Metric  Value  into  the 
box  to  the  right  of  the  item  in  the  Criteria 
Worksheet. 

(5.4)  Repeat  this  process  for  all  items  in  the 
Metric  Summary  Worksection. 

(5.5)  Find  the  Criteria  Value  by  following  the 
algorithm  on  the  worksheet. 

(5.6)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Metrics  as  they 
apply  to  the  system. 

Step  6  -  FACTOR  WORKSHEETS 

(6.1)  Each  numbered  item  in  the  Criteria  Summary 
Worksection  represents  an  individual 
Criterion  Value. 

(6.2)  Find  the  Form  Code  in  the  SOURCE  box  of  the 
Factor  Worksheet. 

(6.3)  Find  the  Criteria  Worksheet  with  the  same 
Form  Code  and  transfer  its  Criteria  Value 
into  the  box  in  the  Score  column  in  the 
Factor  Worksheet. 

(6.4)  Repeat  this  process  for  all  items  in  the 
Criteria  Summary  Worksection. 

(6.5)  Answer  the  Evaluation  Worksection  -  evaluate 
the  products  based  on  the  Criteria  as  they 
apply  to  the  system. 

(6.6)  Find  the  Factor  Value  by  following  the 
algorithm  on  the  Worksheet. 


STEPS  FOR  UPDATE  APPLICATIONS 


Step  1 

Determine  the  update  necessary.  The  Product  Tables 
at  the  end  of  these  instructions  list  the  updates  required, 
and  the  activities  and  products  associated  with  each  up¬ 
date.  Also  review  Section  2.3.2,  "Update  Information 
Requirements",  in  the  General  Instructions  Handbook  for 
further  information  if  necessary. 

Step  2 

Obtain  the  required  products.  The  Products  Tables 
at  the  end  of  these  Instructions  list  the  products  re¬ 
quired  for  each  update  application. 

Step  3 

Reapply  the  Metric,  Criteria,  and  Factor  work¬ 
sheets  by  following  the  procedure  already  outlined  in 
"Steps  for  Completing  Worksheets",  of  these  Instructions. 
"How  to  Use  Score  Chart  sets",  explains  the  procedure  for 
reporting  the  update  scores. 


HOW  TO  USE  SCORE  CHART  SETS 

When  you  apply  worksheets  to  system/subsystem/module, 
ycra  must  apply  some  worksheets  more  than  once  in  the  Com¬ 
puter  Program  Life  Cycle.  Because  of  this,  a  Score  Chart 
set  is  organized  so  that  the  groups  of  worksheet  scores, 
acquired  at  the  times  a  set  of  worksheets  is  applied,  can 
be  recorded  separately:  originals  8  updates. 

The  Score  Chart  set  (at  the  end  of  each  worksheet  set) 
is  composed  of  similar  tree  structure  diagrams.  Each 
Score  Chart  represents  the  complete  hierarchy  of  scores  to| 
be  acquired  during  any  one  application  of  worksheets  sets 
in  the  Life  Cycle. 

When  applying  any  worksheet,  in  order  to  record 
scores  you  must  locate  and  use  the  Score  Chart  that  cor¬ 
responds  to  the  current  phase  in  the  Life  Cycle.  In  a 

Score  Chart  set,  jthe  first  Score  Chart  (labeled  FIRST 
MEASUREMENT)  is  designed  to  receive  the  scores  from  the 
first  application  of  a  worksheet  set  in  the  Life 
Cycle,  and  the  following  charts  (i.e.,  labeled  SECOND 
MEASUREMENT,  etc.)  are  designed  to  receive  the  scores  from 
the  updates. 

Each  data  element  in  the  Metrics  Worksheet  has  a 
unique  number  in  the  Data  Element  Dictionary,  which  is 
listed  in  the  first  column  in  the  Score  Chart.  The  next 
two  columns  of  boxes  in  a  Score  Chart  show  a  worksheet 
set's  individual  data  element  raw  scores  and  its  inter¬ 
preted  response  (Yes/No,  -  1  or  0,  etc.).  The  last  three 
box  columns  map  out  the  Metric,  Criteria  and  Factor  work¬ 
sheets  of  a  set,  each  box  representing  the  total  score  of 
a  corresponding  worksheet. 


Once  you  have  finished  scoring  a  Worksheet  set  and 
located  the  corresponding  Score  Chart  set,  use  the  following 
procedure  for  filling  out  the  Score  Chart: 

Step  1  -  ANSWER  OF  DATA  ELEMENT/DATA  ELEMENT  SCORE 

For  each  data  element  in  a  Metric  Worksheet: 


(1.1) 


(1.2) 


(1.3) 


(1.4) 


Reference  its  number  in  the  "Data  Element 
Number"  Column  in  the  Score  Chart. 

Transfer  the  answers  you  gave  to  the  ques¬ 
tions  in  the  Data  Collection  Worksection  to 
the  corresponding  box  under  the  column  head¬ 
ed  "Answer  of  Data  Element" 

Transfer  the  Scores  of  the  data  element 
from  the  Data  Collection  Worksection  to  its 
corresponding  box  under  column  labeled 
"Data  Element  Score". 

Repeat  this  step  for  all  data  elements  with¬ 
in  this  phase. 


Step  2  -  METRIC  SCORE 

Each  box  in  this  column  represents  the  Score  from 
the  Metric  Worksection  of  a  Metric  Worksheet.  Trans¬ 
fer  the  Metric  Value  from  each  Metric  Worksheet  into 
the  corresponding  boxes  in  the  Score  Chart's  Metric 
Score  Column. 

Step  3  -  CRITERIA  SCORE 

Each  box  in  this  column  represents  the  score  from 
the  Criteria  Wotfksection  of  a  Criteria  Worksheet  into 
the  corresponding  boxes  of  the  Score  Chart's  Criteria 
Score  Column. 
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PRODUCTS 


TRADE¬ 

OFF 

STUDY 


INITIAL  OR 
AUTHENTICATED 
SYSTTM 

AUTHENTICATED 

DEVELOPMENT 

KISER 

DRAFT 

SPEC, 

SPEC 

LEVEL 

PRODUCT 

SEoerr 

FOR 

SPEC 

SPECS 

SPEC 

CPCI 

PDR 

MINUTES 

AND 

ACTION 

ITEM 

RESPONSES 


TRBCNTS 
ANALYSIS 
SHEETS 


PRELIMINARY 

DSSIOf 


SD  PRQUJCrS  FOR  ANALYSIS  PHASE 

Apply  the  Requirement  Analysis  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A. 1  -  Trade-off  Study  Reports 

e  Product  A. 2  -  Initial  or  Authenticated  System  Specifi¬ 

cation  I  Segment  Specification  (if  any) 

•  Product  B.l  -  Authenticated  Development  Specification  for 

each  CPCI 

e  Product  B.2  •  Possible  higher-level  specification,  I CD  and 

changes 

Apply  the  Preliminary  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

e  Product  B.3  -  Parts  of  draft  Product  Specifications  con¬ 
taining  design  approaches  for  each  CPCI 

e  Product  C.  •  PDR  minutes  sad  action  item  responses 


NOTE:  If  these  products  do  not  contain  the  information  requested 
by  the  Data  Element  questions,  other  contractor -supplied 
information  should  be  searched. 
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FUNC¬ 

TIONAL 

FLO* 

CHARTS 


SYSTEM 

CPCI 
TEST 

TEST  |  PRO¬ 
PLANS 


FOR  DESIGN  PHASE 


Apply  the  Detailed  Design  Worksheets  as  the  following 
types  of  products  become  available.  See  note  below. 

•  Product  A.l:  Functional  flowcharts 

•  Product  A. 2:  Detailed  flowcharts 

e  Product  A. 3:  Data  format  descriptions 

e  Product  A. 4:  Descriptions  of  algorithms  not 

previously  prescribed 

e  Product  B.  Preliminary  Product  Specifications, 

including  the  above 

e  Product  D.  CDR  minutes  4  action  item  responses 

Apply  the  Preliminary  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See  note, 

e  Product  C.l:  System,  Segment  and  CPCI 

Test  Plans 

e  Product  C.2:  Preliminary  CPCI  Test  Procedures 

NOTE:  If  these  products  do  not  contain  the  infornation 
requested  by  the  Data  Element  questions,  other 
contractor -supplied  information  should  be  searched. 


rimrritfTrr 


PRODUCTS 


CODE 


ALTERED  PRODUCT  SPECS 


3RXSHEET 
ET  TO  BE 
PPLIED 


IMPLBtENTAnON 

WORKSHEETS 


DETAILED  DESIQf 
WORKSHEETS  (UPDATE) 


ESP  PRODUCTS  FOR  CODING  AND  CHECKING  PHASE 

Apply  the  Implementation  Worksheets  as  the  code 
becomes  available. 

Apply  the  Detailed  Design  Worksheets  for  Update  as 
the  following  types  of  products  become  available.  See 
note  below. 

•  Product  C.  -  Altered  Product  Specifications, 

including  compiler/assembly  listings 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor -supplied  information  should  be  searched 


I 


Apply  the  Preliminary  Design  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

A 

•  Product  1:  Final  CPCZ  Test  Procedures 

e  Product  2:  Segment  (if  any)  and  system* level 
Test  Procedures 

B-0 

e  Product  l:  Test  Reports 

Apply  the  Implementation  Worksheets  for  Update 
as  the  following  types  of  products  become  available.  See 
note  below. 

B-D 

e  Product  2:  Computer  Program  Coding  Changes 
Apply  the  Detailed  Design  Worksheets  for  Update  as  the 
following  types  of  products  become  available.  See  note. 

B-D 

e  Product  3:  Modified  Product  Specification 

e  Product  4:  Possible  high-level  specification, 

I  CD  and  changes.. 

NOTE:  If  these  products  do  not  contain  the  information 
requested  by  the  Data  Element  questions,  other 
contractor-supplied  information  should  be  searched. 
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□□ 


OOMJNI  CATION  CCM*DNALITY  CHECKLIST 

Form  Code:  IpRAM.l 


LIFE  CYCLE  PHASE:  SOURCE (S): 

REQUIREMENTS  ANALYSIS 


nATA  rniXECTiON  WORKSECTION: 

1.0  Cnwamication  with  Other  System  (27) 

1.1  Is  there  a  definitive  statement  of 
the  requixeneot  for  caaamication 
with  other  systens? 

Yes  •  1,  No  «  0 


□□ 


DATA  COMONALITY  CHECKLIST 


Form  Code:  IpRAM.2 


LIFE  OCXS  PHASE: 

REQUIREMENT  ANALYSIS 


DATA  COLLECTION  WCRK5ECTI0N: 


SCORE 


tatloc  for  finemmi  cation  (28) 


Is  there  a  definitive  statement  of 
the  requireuent  for  standard  data 
representation  for  coBwinication 
with  other  systeas? 

Yes  •  1,  No  •  0 


II.  METRIC  WJRKSECnON: 


.. _ _  _  _ Sian  of  Above  Scores 

Metric  Value  ■  nr — gr==v. 


III.  EVALUATION 


(1-10) 


Rhat  is  your  evaluation  of  the 
on  the  data  eleents  above? 

( |  If  you  are  unable  to  evaluate) 


PREPARED  BY:. 
WOl: _ 


APPROVED  BY: 
DATE: _ 


Ip-17 


□□ 


COMMUNICATION  CCMONALITY 


8 


TOEPAHED  BY:  APPROVED  BY:, 

EKIE: _ '  DA3E: _ 
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□□ 


interoperability 


Complete  tJh<«  soon  chart  if  an  application  of  this  set  of  worksheets  is  appro¬ 
priate  when  products  of  Al.  A2,  M  and  B2  in  Analysis  Phase  become  available. 


Data  Answer  of  Data 

Element  Data  FI  nr  Metric  Criteria  Factor 

Natter  Oiemt  Score  Score  Score  Score 


FIRST  MEASURBffiNT  Date 


Conplete  this  score  chart  if  a  second  application  of  this  set  of  worksheets  is 
appropriate  for  update. 


Data  Answer  of  Data 
Element  Data  Element 

Nuaber  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


SECOND  MEASUREMENT  Date 
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i<r>y  1-  IWP.  WP 


C0W4UNI CATION  CCMdONALITY  CHECKLIST 

Foss  Code:  IpPEM.l 

LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 

SOURCE(S): 

□□□ 

NAfrE:  _ 

I.  DATA  COLLECTION  WORKSECTION: 

adards  (46) 

ds  been  established,  r- 1— 

?  Y  N 

(47) 

n 

SOORE 

1.0  Gt—»mi  ration  Protocol  St* 

1.1  Have  protocol  standan 
and  are  they  followed 

Yes  -  1,  NO  •  0 

2.0  Module  Interface  for  Input 

2.1  Number  of  nodules  use 
.  fro*  other  system. 

Score  ■  1  +  2.1 

□  □ 

II.  METRIC  WORKSECTION: 

Metric  Vi 

Sum  of  Above  Scores 

□ 

iluc  -  no.  of  Data  Elements 

III.  EVALUATION  WORKSECTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elcamts  above? 

(1-10)  (0  If  you  are  unable  to  evaluate) 

IV.  INSPECTOR'S  GOMENTS: 

PREPARED  BY: 


APPROVED  BY: 


DAIS: 


DA3E: 


DATA  COfrCNALITY  CHECKLIST 


Fora  Code:  I 


LIFE  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SOURS  (S) 


DATA  COLLECnON  WCRKSECTION: 

1.0  Translation  Standard!  (49) 

1.1  Have  standard  data  representations 
or  translation  standards  between 
representations  been  established, 
aid  are  they  being  eoeplied  with? 

Yes  -  1,  No  •  0 

2.0  Translation  Perforaance  (50) 

2.1  Nueber  of  nodules  used  to  perfora 
translations. 


Score  •  1  12.1 


II.  METRIC  VORKSEmON: 


III*  EVALUATION  WCRKSECTICN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  data  elements  above? 
(1-10) _  (9  If  you  are  unable  to  evaluate) 


Metric  Value  ■ 


Sum  of  Above  Scores 


□□ 


DATA  CXMGNALITY 


Font  Code:  IpPDC.l 


II. 

CRITERIA  WQRXSECnON: 

rvTrm-ri  Valiai  0t.  A607?  5cores 

□ 

Criteria  value  ^jo.  of  Metrics 

III*  EVALUATION  MORKSBCTICN:  What  is  your  evaluation  of  the 
reviewed  products  ba35d  on  the  aetrics  above? 

(1-103  (jl  If  you  are  unable  to  evaluate) 


□□ 


COMUNICATION  OOMDNALITY 


INTEROPERABILITY 


Fora  code:  IpPDF.l 


!  CYCLE  PHASE: 

PRELIMINARY  DESIGN 


SYSIBf 


NA*E: 


SOURCE  (S) : 
IpPDC.l 


IpPDC.2 


|  SOBSYSOM  - 

j  - 

CRITERIA  SUMMARY  WCRXSECTICN: 


score 


1.  Data  Co— onality 

2.  Caaaini ration  Coaaonality 


II. 

FACTOR  WQRKSECTXON: 

O 

- _ _ _ Sub  of  Above  Scores 

***orV1m  -  fe.'  bTCfiteHa - 

INTEROPERABILITY 


Conplete  this  score  chart  if  an  application  of  this  set  of  worksheets  is  appro* 
priate  when  products  of  B3  and  C  in  Analysis  Phase  becooe  available. 


Data  Answer  of  Oats 

Haunt  Data  Flenent  Metric  Criteria  Factor 

Maher  El— at  Score  Score  Scar*  Score 


Conplete  this  score  chart  if  a  second  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Q  and  C2  in  Design  Phase  bee— 
available. 


INTHKOPERABILTOT 


Complete  this  score  chert  if  a  third  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2,  and  B1  in  Test  and  Integration 
Phase  become  available. 


Data  Answer  of  Data 

EUmut  Data  Eli— nt  Metric  Criteria  Actor 

Ele— nt  Score  Score  Score  Score 


Complete  this  score  chart  if  a  fourth  application  of  this  set  of  worksheets  is 
appropriate  for  update  when  products  of  Al,  A2  and  B-Dl  in  Test  and  Integration 
Phase  became  available. 


Data  Answer  of  Data 

Ele—nt  Data  Elea— t  Metric  Criteria  Factor 

Nuebtr  Fluent  Score  Score  Score  Score 


MODULAR  IMPLEMENTATION  KEASURE 


i  CYCLE  PHASE: 


SOURCE  (S): 


DETAIL  DESICH 


[  SYSTEM 

NAIfi: 

1  SUBSYSTEM 

|  M33ULE 

Fora  Code: 


DATA  COLLECTION  KMSECTICN: 

1.0  Controlling  Par— tors  (87) 

1.1  Nuaber  of  celling  sequence  perm* 
asters  that  ere  control  parameters. 

1.2  Nuaber  of  celling  sequence  para- 
aeters. 

Score  «o+o 

2.0  Controlling  Input  (88) 

2.1  Is  input  pessed  as  calling  sequenre 
paraaeters? 


Yes  •  1,  No  •  0 
3.0  Gontrollinp  Output  (89) 


3.1  Is  output  date  passed  back  to 
calling  nodule? 


Yes  ■  1,  No  ■  0 
4.0  Controlling  (90) 


4.1  Is  control  returned  to  calling 
nodule? 


□ 

□ 


Yes  -  1,  No  •  0 


□□ 


□□ 


MODULARITY 


Porn  Cod*:  IpDDC.l 


CYCLE  PHASE: 

DETAIL  DESIQ4 


S0(KZ(S): 

IpDDM.l 


METRIC  SUM4ARY 


1.  Modular  Implementation  Measure 


II.  CRITERIA  WORKSECnON: 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION  WORKSBCTION:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  ae tries  above? 

(1-10)  .  (jl  If  you  are  unable  to  evaluate) 


INTEROPERABILITY 


Form  Code:  I 


CYCLE  PHASE: 

DETAIL  DESIOi 


SOGRCECS]: 

IpCDC.l 


I.  CRITERIA  SIM4AEY  WCRKSECTION: 
1.  Modularity 


II.  FACTOR  WORJSECTION: 


Factor  Value  • 


Sum  of  Above  Scores 


III.  EVALUATION  WORXSliCriCN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  (Jf  If  yc>u  are  unable  to  evaluate) 


BY 

APHOVED 

BY 

DAIS: 

INTEROPERABILITY 


Complete  this  score  chart  if  an  application  of  this  set  of  worksheets  is  appro 
priate  when  products  of  Al,  A2,  A3,  A4,  B  and  D  in  Design  phase  beocne  availa¬ 
ble. 


Data  Answer  of  Data 

El  went  Data  Element 

Maher  Element  Score 


Metric  Criteria  Factor 

Score  Score  Score 


FIRST  KEASURIMENT 


Date 


Gonplete  this  score  chart  is  a  second  (or  iterative) 
of  worksheets  is  appropriate  for  update  when  product 
out  phase  becomes  available. 


application  of  this  set 
of  C  in  coding  and  check' 


Data  Answer  of  Data 

Element  Data  Element 
Nunber  Element  Score 


Metric  Criteria  Factor 
Score  Score  Score 


C13#§  Jjl] 


MEASUREMENT 


Date 


INTEROPERABILITY 


Complete  this  score  chart  if  a  third  application  of  this  set  of  worksheets  is 
appropriate  for  qpdate  when  products  of  B-D3,  B-D4  is  Test  and  Integration 
phase  become  available. 


Data  Answer  of  Data 

Element  Data  Element 
Number  Element  Score 


Metric  Criteria  Factor 
Score  Score  Score 


THIRD  MEASUREMENT 


Date 


MODULAR 

IMPLEMENTATION  MEASURE 

For*  Code:  IpIM4.1 

LIFE  CYCLE  PHASE: 

IMPLEMENTATION 

SOURCE (S) : 

NNC: 


DATA  OOLI£CnON  WORKSECTICN: 

1.0  Controlling  Parameters  (87) 

1.1  Nuaber  of  calling  sequence  para- 
meters  that  are  control  parameters . 

1.2  Nusber  of  calling  sequence  para- 
®eters. 

Score  ■  1 1.1 1  +  1.2 

2.0  Controlling  Input  (88) 

2.1  Is  input  passed  as  calling  sequence 
parameters? 

Yes  -  1,  No  -  0 

3.0  Controlling  Output  (89) 

3.1  Is  output  data  passed  back  to 
calling  module? 

Yes  -  1.  No  -  0 

4.0  Controlling  Return  (90) 

4.1  Is  control  returned  to  calling 
module? 

Yes  ■  1,  No  ■  0 


II.  METRIC  NORKSECTION: 


Metric  Value  » 


Sum  of  Above  Scores 


III.  EVALUATION  WCRK5ECTICN:  What  is  your  evaluation  of  the 
reviewed!  products  based  on  the  data  elements  above? 
(1-10) _  (9  If  you  are  unable  to  evaluate) 


BY 


MODULARITY 


Form  Code: 


CYCLE  PHASE: 

'  IMPLBENTATICN 


SOURCE (S): 
IpIMM.l 


METRIC  SUMMARY  WORKSECTION: 


1.  Modular  Implementation  Measure 


II.  CRITERIA  MORXSECTION: 


Criteria  Value 


Sum  of  Above  Scores 


III.  EVALUATION 


(1-10). 


What  is  your  evaluation  of  the 
an  the  metrics  above? 

(|  If  you  are  unable  to  evaluate) 


» 


INTEROPERABILITY 


Fom  Cock:  IpIMF.l 


LIFE  CYCLE  PHASE: 

IMPLEMENTATION 

S00RGBCS): 

IpDC.l 

□  sysiw 

□  SOBSYSTBf 

□  MODULE 

I.  CRITERIA  SUMMARY  WGRJCSECTICN: 

1 .  Modularity 

\m^m\ 

□ 

II.  FACTOR  HCRXSECnGN: 

_ _ , _ Sub  of  Above  Scores 

o 

^  No.  of  Criteria 

III.  EVALUATION  WGRXSECTIGN:  What  is  your  evaluation  of  the 
reviewed  products  based  on  the  criteria  above? 

(1-10) _  ( 9  If  you  are  unable  to  evaluate) 

PREPARED  BY: 

APPROVED  BY: 

Ip-37 


INTEROPERABILITY 


Conplete  this  score  chart  if  an  application  of  this  set  of  worksheets  is 
appropriate  when  products  of  A  and  B  in  Coding  and  Qwckout  Phase  becaae 
available. 


Data  Answer  of 
Eleaent  Data 
Number  Eleaent 


Data 

Eleaent 

Score 


Metric  Criteria  Factor 

Score  Score  Score 


87 

87 

88 

89 

90 


FIRST  KCASUREMENT  Date 


Coenlete  this  score  chart  if  a  second  application  of  this  set  of  worksheets 
is  appropriate  for  update  when  products  of  B-D2  in  Test  and  Integration 
Phase  beccnes  available. 

Data  Answer  of  Data 

Eleaent  Data  Eleaent  Metric  Criteria  Factor 

Nunber  Eleaent  Score  Score  Scotc  Score 


INTEROPERABILITY 


Ccnplete  this  score  chart  if  a  third  application  of  this  set  of  worksheets 
is  appropriate  for  update. 

Data  Answer  of  Data 

Element  Data  Element  Metric  Criteria  Factor 

Number  Element  Score  Score  Score  Score 


